[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:13:55.111307 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:40673
I20260709 13:13:55.113133 10257 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:13:55.114280 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:55.131784 10266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:55.132105 10264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:55.132051 10263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:56.258217 10265 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1126 milliseconds
I20260709 13:13:56.258342 10257 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 13:13:56.259588 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:56.259809 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:56.259994 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602836259968 us; error 0 us; skew 500 ppm
I20260709 13:13:56.260798 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:56.265707 10257 webserver.cc:533] Webserver started at http://127.10.4.126:41897/ using document root <none> and password file <none>
I20260709 13:13:56.266523 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:56.266726 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:56.267148 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:56.272866 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "f7e8bd805c6e404da4e56529d24db1cf"
format_stamp: "Formatted at 2026-07-09 13:13:56 on dist-test-slave-gkm9"
I20260709 13:13:56.280345 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:13:56.285354 10272 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:56.286422 10257 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:13:56.286743 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "f7e8bd805c6e404da4e56529d24db1cf"
format_stamp: "Formatted at 2026-07-09 13:13:56 on dist-test-slave-gkm9"
I20260709 13:13:56.287057 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:56.351315 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:56.352950 10257 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:13:56.353390 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:56.422773 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:40673
I20260709 13:13:56.422850 10323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:40673 every 8 connection(s)
I20260709 13:13:56.430186 10324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:56.451236 10324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf: Bootstrap starting.
I20260709 13:13:56.456833 10324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:56.458535 10324 log.cc:826] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf: Log is configured to *not* fsync() on all Append() calls
I20260709 13:13:56.462860 10324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf: No bootstrap required, opened a new log
I20260709 13:13:56.483273 10324 raft_consensus.cc:359] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e8bd805c6e404da4e56529d24db1cf" member_type: VOTER }
I20260709 13:13:56.483862 10324 raft_consensus.cc:385] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:56.484129 10324 raft_consensus.cc:740] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e8bd805c6e404da4e56529d24db1cf, State: Initialized, Role: FOLLOWER
I20260709 13:13:56.484998 10324 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e8bd805c6e404da4e56529d24db1cf" member_type: VOTER }
I20260709 13:13:56.485457 10324 raft_consensus.cc:399] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:13:56.485694 10324 raft_consensus.cc:493] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:13:56.485967 10324 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:56.489879 10324 raft_consensus.cc:515] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e8bd805c6e404da4e56529d24db1cf" member_type: VOTER }
I20260709 13:13:56.490569 10324 leader_election.cc:304] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7e8bd805c6e404da4e56529d24db1cf; no voters: 
I20260709 13:13:56.492252 10324 leader_election.cc:290] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:13:56.492569 10327 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:56.494959 10327 raft_consensus.cc:697] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 1 LEADER]: Becoming Leader. State: Replica: f7e8bd805c6e404da4e56529d24db1cf, State: Running, Role: LEADER
I20260709 13:13:56.495983 10327 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e8bd805c6e404da4e56529d24db1cf" member_type: VOTER }
I20260709 13:13:56.497043 10324 sys_catalog.cc:565] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:13:56.506639 10328 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7e8bd805c6e404da4e56529d24db1cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e8bd805c6e404da4e56529d24db1cf" member_type: VOTER } }
I20260709 13:13:56.507153 10328 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [sys.catalog]: This master's current role is: LEADER
I20260709 13:13:56.508504 10329 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e8bd805c6e404da4e56529d24db1cf. Latest consensus state: current_term: 1 leader_uuid: "f7e8bd805c6e404da4e56529d24db1cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e8bd805c6e404da4e56529d24db1cf" member_type: VOTER } }
I20260709 13:13:56.509083 10329 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [sys.catalog]: This master's current role is: LEADER
I20260709 13:13:56.510130 10336 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:13:56.521485 10336 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:13:56.523584 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:13:56.536507 10336 catalog_manager.cc:1383] Generated new cluster ID: d2da2ca3bbe14d3a912857a39f0b5b83
I20260709 13:13:56.536746 10336 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:13:56.565069 10336 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:13:56.567229 10336 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:13:56.588408 10336 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf: Generated new TSK 0
I20260709 13:13:56.589315 10336 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:13:56.654285 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:56.662147 10346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:56.664762 10347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:56.667102 10349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:56.667979 10257 server_base.cc:1061] running on GCE node
I20260709 13:13:56.668910 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:56.669162 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:56.669368 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602836669351 us; error 0 us; skew 500 ppm
I20260709 13:13:56.670162 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:56.674067 10257 webserver.cc:533] Webserver started at http://127.10.4.65:33063/ using document root <none> and password file <none>
I20260709 13:13:56.674759 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:56.674999 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:56.675323 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:56.676791 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "cd0332521e0b4db09972c056204d24e3"
format_stamp: "Formatted at 2026-07-09 13:13:56 on dist-test-slave-gkm9"
I20260709 13:13:56.683475 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:13:56.688249 10354 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:56.689051 10257 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:13:56.689329 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "cd0332521e0b4db09972c056204d24e3"
format_stamp: "Formatted at 2026-07-09 13:13:56 on dist-test-slave-gkm9"
I20260709 13:13:56.689616 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:56.711967 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:56.713374 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:56.715931 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:56.720141 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:56.720402 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:56.720676 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:56.720845 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:56.812767 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:34327
I20260709 13:13:56.812902 10416 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:34327 every 8 connection(s)
I20260709 13:13:56.819494 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:56.829201 10421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:56.832581 10422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:56.836745 10257 server_base.cc:1061] running on GCE node
W20260709 13:13:56.837328 10424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:56.838539 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:56.838753 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:56.838922 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602836838903 us; error 0 us; skew 500 ppm
I20260709 13:13:56.839537 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:56.842129 10257 webserver.cc:533] Webserver started at http://127.10.4.66:42883/ using document root <none> and password file <none>
I20260709 13:13:56.842610 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:56.842794 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:56.843048 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:56.844379 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "c6c412495aa9434b800110f744218c0a"
format_stamp: "Formatted at 2026-07-09 13:13:56 on dist-test-slave-gkm9"
I20260709 13:13:56.847438 10417 heartbeater.cc:344] Connected to a master server at 127.10.4.126:40673
I20260709 13:13:56.847908 10417 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:56.849027 10417 heartbeater.cc:507] Master 127.10.4.126:40673 requested a full tablet report, sending...
I20260709 13:13:56.850310 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:13:56.852437 10289 ts_manager.cc:194] Registered new tserver with Master: cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327)
I20260709 13:13:56.855123 10429 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:56.855188 10289 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47288
I20260709 13:13:56.856093 10257 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:13:56.856432 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "c6c412495aa9434b800110f744218c0a"
format_stamp: "Formatted at 2026-07-09 13:13:56 on dist-test-slave-gkm9"
I20260709 13:13:56.856823 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:56.878276 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:56.879585 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:56.881916 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:56.885113 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:56.885289 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:56.885535 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:56.885677 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:56.975613 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:42071
I20260709 13:13:56.976018 10491 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:42071 every 8 connection(s)
I20260709 13:13:56.981954 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:56.989672 10495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:56.991842 10496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:56.995373 10498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:56.996758 10492 heartbeater.cc:344] Connected to a master server at 127.10.4.126:40673
I20260709 13:13:56.997092 10492 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:56.997896 10257 server_base.cc:1061] running on GCE node
I20260709 13:13:56.997998 10492 heartbeater.cc:507] Master 127.10.4.126:40673 requested a full tablet report, sending...
I20260709 13:13:56.998993 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:56.999260 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:56.999487 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602836999466 us; error 0 us; skew 500 ppm
I20260709 13:13:57.000156 10289 ts_manager.cc:194] Registered new tserver with Master: c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:57.000237 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:57.001713 10289 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47290
I20260709 13:13:57.003314 10257 webserver.cc:533] Webserver started at http://127.10.4.67:38447/ using document root <none> and password file <none>
I20260709 13:13:57.003988 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:57.004233 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:57.004529 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:57.005986 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "98ef1ab89307452d8cd1dfa48d30564c"
format_stamp: "Formatted at 2026-07-09 13:13:57 on dist-test-slave-gkm9"
I20260709 13:13:57.011770 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:13:57.015946 10503 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:57.016857 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:13:57.017127 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "98ef1ab89307452d8cd1dfa48d30564c"
format_stamp: "Formatted at 2026-07-09 13:13:57 on dist-test-slave-gkm9"
I20260709 13:13:57.017390 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:57.031284 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:57.032488 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:57.033882 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:57.036684 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:57.036852 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:57.037086 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:57.037223 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:57.122947 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:38229
I20260709 13:13:57.123042 10565 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:38229 every 8 connection(s)
I20260709 13:13:57.142347 10566 heartbeater.cc:344] Connected to a master server at 127.10.4.126:40673
I20260709 13:13:57.142692 10566 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:57.143505 10566 heartbeater.cc:507] Master 127.10.4.126:40673 requested a full tablet report, sending...
I20260709 13:13:57.145762 10289 ts_manager.cc:194] Registered new tserver with Master: 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:57.145885 10257 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013933146s
I20260709 13:13:57.147095 10289 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47296
I20260709 13:13:57.200178 10289 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47298:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:13:57.203843 10289 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:13:57.306744 10382 tablet_service.cc:1511] Processing CreateTablet for tablet 7eba7c08f8294327b314adc5ac590b05 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:13:57.307653 10380 tablet_service.cc:1511] Processing CreateTablet for tablet 73845cf648b3497c9309033a7379b5be (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:13:57.308387 10382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eba7c08f8294327b314adc5ac590b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.309278 10380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73845cf648b3497c9309033a7379b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.310673 10381 tablet_service.cc:1511] Processing CreateTablet for tablet 030176ba7ebe4dc381240a987d5fddb4 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:13:57.312083 10381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 030176ba7ebe4dc381240a987d5fddb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.316777 10529 tablet_service.cc:1511] Processing CreateTablet for tablet 73845cf648b3497c9309033a7379b5be (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:13:57.317155 10531 tablet_service.cc:1511] Processing CreateTablet for tablet 7eba7c08f8294327b314adc5ac590b05 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:13:57.318195 10529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73845cf648b3497c9309033a7379b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.318858 10530 tablet_service.cc:1511] Processing CreateTablet for tablet 030176ba7ebe4dc381240a987d5fddb4 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:13:57.320154 10531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eba7c08f8294327b314adc5ac590b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.326478 10530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 030176ba7ebe4dc381240a987d5fddb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.332120 10379 tablet_service.cc:1511] Processing CreateTablet for tablet e5b8957e7eef43a79f798a63434e9b39 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:13:57.336845 10378 tablet_service.cc:1511] Processing CreateTablet for tablet 43b36a3a288649daa015b4bd91c84674 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:13:57.336189 10528 tablet_service.cc:1511] Processing CreateTablet for tablet e5b8957e7eef43a79f798a63434e9b39 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:13:57.344475 10528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b8957e7eef43a79f798a63434e9b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.340193 10378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b36a3a288649daa015b4bd91c84674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.346858 10379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b8957e7eef43a79f798a63434e9b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.370036 10527 tablet_service.cc:1511] Processing CreateTablet for tablet 43b36a3a288649daa015b4bd91c84674 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:13:57.374082 10527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b36a3a288649daa015b4bd91c84674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.373204 10380 tablet_service.cc:1511] Processing CreateTablet for tablet 597045aaee954382839393fe2ac62d46 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:13:57.374680 10584 tablet_bootstrap.cc:492] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:57.378543 10380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597045aaee954382839393fe2ac62d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.399706 10526 tablet_service.cc:1511] Processing CreateTablet for tablet 597045aaee954382839393fe2ac62d46 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:13:57.401139 10526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597045aaee954382839393fe2ac62d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.411321 10457 tablet_service.cc:1511] Processing CreateTablet for tablet 7eba7c08f8294327b314adc5ac590b05 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:13:57.411562 10456 tablet_service.cc:1511] Processing CreateTablet for tablet 030176ba7ebe4dc381240a987d5fddb4 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:13:57.412464 10382 tablet_service.cc:1511] Processing CreateTablet for tablet c3836d83f3cc483b8efb7ebd8e67dfd1 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:13:57.412938 10456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 030176ba7ebe4dc381240a987d5fddb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.413628 10382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3836d83f3cc483b8efb7ebd8e67dfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.414680 10452 tablet_service.cc:1511] Processing CreateTablet for tablet 597045aaee954382839393fe2ac62d46 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:13:57.415920 10452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597045aaee954382839393fe2ac62d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.415621 10451 tablet_service.cc:1511] Processing CreateTablet for tablet c3836d83f3cc483b8efb7ebd8e67dfd1 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:13:57.416811 10451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3836d83f3cc483b8efb7ebd8e67dfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.417971 10455 tablet_service.cc:1511] Processing CreateTablet for tablet 73845cf648b3497c9309033a7379b5be (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:13:57.412604 10454 tablet_service.cc:1511] Processing CreateTablet for tablet e5b8957e7eef43a79f798a63434e9b39 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:13:57.422477 10381 tablet_service.cc:1511] Processing CreateTablet for tablet 0a54d74c40c043b8a6daab30bc33d4fd (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:13:57.424983 10453 tablet_service.cc:1511] Processing CreateTablet for tablet 43b36a3a288649daa015b4bd91c84674 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:13:57.426277 10381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a54d74c40c043b8a6daab30bc33d4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.429277 10455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73845cf648b3497c9309033a7379b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.440379 10454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b8957e7eef43a79f798a63434e9b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.446868 10457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eba7c08f8294327b314adc5ac590b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.457540 10453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43b36a3a288649daa015b4bd91c84674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.457955 10585 tablet_bootstrap.cc:492] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:57.476357 10584 tablet_bootstrap.cc:654] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.474176 10531 tablet_service.cc:1511] Processing CreateTablet for tablet c3836d83f3cc483b8efb7ebd8e67dfd1 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:13:57.486793 10531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3836d83f3cc483b8efb7ebd8e67dfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.487078 10526 tablet_service.cc:1511] Processing CreateTablet for tablet 0a54d74c40c043b8a6daab30bc33d4fd (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:13:57.489945 10526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a54d74c40c043b8a6daab30bc33d4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.490537 10450 tablet_service.cc:1511] Processing CreateTablet for tablet 0a54d74c40c043b8a6daab30bc33d4fd (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:13:57.492920 10450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a54d74c40c043b8a6daab30bc33d4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.496958 10586 tablet_bootstrap.cc:492] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:57.512545 10378 tablet_service.cc:1511] Processing CreateTablet for tablet 1340349398384547b61df98ecaf81f90 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
W20260709 13:13:57.513357 10337 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:57.514120 10378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1340349398384547b61df98ecaf81f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.523102 10451 tablet_service.cc:1511] Processing CreateTablet for tablet 1340349398384547b61df98ecaf81f90 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:13:57.524308 10451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1340349398384547b61df98ecaf81f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.548628 10586 tablet_bootstrap.cc:654] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.548404 10528 tablet_service.cc:1511] Processing CreateTablet for tablet 1340349398384547b61df98ecaf81f90 (DEFAULT_TABLE table=test-workload [id=7317219a34dc446b99a980acde9f02f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:13:57.550364 10528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1340349398384547b61df98ecaf81f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:57.551095 10584 tablet_bootstrap.cc:492] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:57.552320 10585 tablet_bootstrap.cc:654] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.551623 10584 ts_tablet_manager.cc:1403] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.182s	user 0.013s	sys 0.004s
I20260709 13:13:57.564110 10586 tablet_bootstrap.cc:492] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:57.564654 10586 ts_tablet_manager.cc:1403] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.003s
I20260709 13:13:57.565595 10585 tablet_bootstrap.cc:492] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:57.566015 10585 ts_tablet_manager.cc:1403] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.000s
I20260709 13:13:57.568097 10586 raft_consensus.cc:359] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.568975 10586 raft_consensus.cc:385] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.568868 10584 raft_consensus.cc:359] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.569334 10586 raft_consensus.cc:740] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.569603 10584 raft_consensus.cc:385] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.569897 10584 raft_consensus.cc:740] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.570303 10586 consensus_queue.cc:260] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.572160 10585 raft_consensus.cc:359] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.572885 10585 raft_consensus.cc:385] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.573168 10585 raft_consensus.cc:740] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.570591 10584 consensus_queue.cc:260] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.577031 10585 consensus_queue.cc:260] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.581426 10417 heartbeater.cc:499] Master 127.10.4.126:40673 was elected leader, sending a full tablet report...
I20260709 13:13:57.582489 10584 ts_tablet_manager.cc:1434] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:13:57.583702 10584 tablet_bootstrap.cc:492] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:57.603606 10585 ts_tablet_manager.cc:1434] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.037s	user 0.018s	sys 0.013s
I20260709 13:13:57.606632 10584 tablet_bootstrap.cc:654] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.607789 10566 heartbeater.cc:499] Master 127.10.4.126:40673 was elected leader, sending a full tablet report...
I20260709 13:13:57.608263 10585 tablet_bootstrap.cc:492] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:57.616394 10586 ts_tablet_manager.cc:1434] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.051s	user 0.024s	sys 0.023s
I20260709 13:13:57.616907 10492 heartbeater.cc:499] Master 127.10.4.126:40673 was elected leader, sending a full tablet report...
I20260709 13:13:57.624184 10586 tablet_bootstrap.cc:492] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:57.617041 10585 tablet_bootstrap.cc:654] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:57.632982 10567 tablet.cc:2406] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:13:57.633543 10586 tablet_bootstrap.cc:654] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.637667 10585 tablet_bootstrap.cc:492] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:57.638137 10585 ts_tablet_manager.cc:1403] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.000s
I20260709 13:13:57.641144 10585 raft_consensus.cc:359] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:57.641978 10585 raft_consensus.cc:385] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.642287 10585 raft_consensus.cc:740] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.643211 10585 consensus_queue.cc:260] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:57.645512 10585 ts_tablet_manager.cc:1434] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:13:57.646453 10585 tablet_bootstrap.cc:492] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:57.652997 10584 tablet_bootstrap.cc:492] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:57.653447 10584 ts_tablet_manager.cc:1403] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.070s	user 0.004s	sys 0.009s
I20260709 13:13:57.653493 10585 tablet_bootstrap.cc:654] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.656287 10584 raft_consensus.cc:359] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:57.656982 10584 raft_consensus.cc:385] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.657279 10584 raft_consensus.cc:740] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.658156 10584 consensus_queue.cc:260] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:57.658924 10586 tablet_bootstrap.cc:492] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:57.659785 10586 ts_tablet_manager.cc:1403] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.013s
I20260709 13:13:57.661525 10585 tablet_bootstrap.cc:492] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:57.662032 10585 ts_tablet_manager.cc:1403] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:13:57.669943 10584 ts_tablet_manager.cc:1434] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:13:57.670063 10586 raft_consensus.cc:359] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.670208 10585 raft_consensus.cc:359] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.670972 10585 raft_consensus.cc:385] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.670922 10586 raft_consensus.cc:385] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.671509 10585 raft_consensus.cc:740] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.671551 10586 raft_consensus.cc:740] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.672497 10586 consensus_queue.cc:260] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.672874 10585 consensus_queue.cc:260] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.674882 10585 ts_tablet_manager.cc:1434] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:13:57.676429 10585 tablet_bootstrap.cc:492] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:57.676779 10586 ts_tablet_manager.cc:1434] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.006s
I20260709 13:13:57.677659 10586 tablet_bootstrap.cc:492] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:57.682767 10585 tablet_bootstrap.cc:654] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.689978 10586 tablet_bootstrap.cc:654] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.695509 10584 tablet_bootstrap.cc:492] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:57.698297 10585 tablet_bootstrap.cc:492] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:57.698789 10585 ts_tablet_manager.cc:1403] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 13:13:57.700100 10586 tablet_bootstrap.cc:492] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:57.700546 10586 ts_tablet_manager.cc:1403] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.000s
I20260709 13:13:57.701766 10585 raft_consensus.cc:359] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.702502 10585 raft_consensus.cc:385] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.702852 10584 tablet_bootstrap.cc:654] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.702796 10585 raft_consensus.cc:740] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.703476 10586 raft_consensus.cc:359] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.704202 10586 raft_consensus.cc:385] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.703856 10585 consensus_queue.cc:260] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.704509 10586 raft_consensus.cc:740] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.706136 10585 ts_tablet_manager.cc:1434] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:13:57.707182 10585 tablet_bootstrap.cc:492] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:57.713354 10585 tablet_bootstrap.cc:654] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.718099 10584 tablet_bootstrap.cc:492] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:57.718547 10584 ts_tablet_manager.cc:1403] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 13:13:57.718357 10586 consensus_queue.cc:260] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.721028 10586 ts_tablet_manager.cc:1434] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:13:57.721956 10586 tablet_bootstrap.cc:492] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:57.728796 10586 tablet_bootstrap.cc:654] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.730228 10592 raft_consensus.cc:493] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.730732 10592 raft_consensus.cc:515] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.733628 10592 leader_election.cc:290] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:57.734398 10584 raft_consensus.cc:359] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.735113 10584 raft_consensus.cc:385] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.735390 10584 raft_consensus.cc:740] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.736158 10584 consensus_queue.cc:260] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.745445 10584 ts_tablet_manager.cc:1434] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:13:57.747833 10585 tablet_bootstrap.cc:492] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:57.748240 10586 tablet_bootstrap.cc:492] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:57.748310 10585 ts_tablet_manager.cc:1403] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.012s
I20260709 13:13:57.748720 10586 ts_tablet_manager.cc:1403] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.007s
I20260709 13:13:57.751276 10586 raft_consensus.cc:359] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.751978 10584 tablet_bootstrap.cc:492] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:57.752025 10586 raft_consensus.cc:385] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.752557 10586 raft_consensus.cc:740] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.758180 10392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:57.759018 10584 tablet_bootstrap.cc:654] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.751161 10585 raft_consensus.cc:359] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.759156 10392 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 0.
I20260709 13:13:57.759479 10585 raft_consensus.cc:385] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.759797 10585 raft_consensus.cc:740] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.760493 10585 consensus_queue.cc:260] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.761332 10507 leader_election.cc:304] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:57.762547 10592 raft_consensus.cc:2804] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.762868 10592 raft_consensus.cc:493] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.763209 10592 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.761363 10586 consensus_queue.cc:260] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.770692 10590 raft_consensus.cc:493] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.771196 10590 raft_consensus.cc:515] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.773521 10585 ts_tablet_manager.cc:1434] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 13:13:57.774659 10585 tablet_bootstrap.cc:492] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:57.775208 10590 leader_election.cc:290] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:57.787163 10592 raft_consensus.cc:515] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.799590 10591 raft_consensus.cc:493] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.800206 10591 raft_consensus.cc:515] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:57.802778 10590 raft_consensus.cc:493] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.803277 10590 raft_consensus.cc:515] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.805068 10592 leader_election.cc:290] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:57.806990 10467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:57.808720 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a"
W20260709 13:13:57.810683 10505 leader_election.cc:343] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6c412495aa9434b800110f744218c0a (127.10.4.66:42071): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.811982 10590 leader_election.cc:290] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
W20260709 13:13:57.812021 10505 leader_election.cc:343] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c6c412495aa9434b800110f744218c0a (127.10.4.66:42071): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.814124 10392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:57.817731 10392 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.814110 10591 leader_election.cc:290] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:57.821065 10584 tablet_bootstrap.cc:492] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:57.821631 10584 ts_tablet_manager.cc:1403] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.070s	user 0.006s	sys 0.013s
I20260709 13:13:57.825362 10584 raft_consensus.cc:359] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.826191 10584 raft_consensus.cc:385] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.826493 10584 raft_consensus.cc:740] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.826740 10392 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 1.
I20260709 13:13:57.827675 10584 consensus_queue.cc:260] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.828298 10586 ts_tablet_manager.cc:1434] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.014s
I20260709 13:13:57.828655 10591 raft_consensus.cc:493] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.829403 10586 tablet_bootstrap.cc:492] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:57.829159 10591 raft_consensus.cc:515] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.843734 10591 leader_election.cc:290] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:57.851447 10585 tablet_bootstrap.cc:654] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.855070 10608 raft_consensus.cc:493] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.855772 10608 raft_consensus.cc:515] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.860862 10586 tablet_bootstrap.cc:654] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.868956 10608 leader_election.cc:290] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:57.870987 10584 ts_tablet_manager.cc:1434] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.049s	user 0.018s	sys 0.001s
I20260709 13:13:57.872045 10584 tablet_bootstrap.cc:492] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:57.865665 10507 leader_election.cc:304] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: c6c412495aa9434b800110f744218c0a
I20260709 13:13:57.877557 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597045aaee954382839393fe2ac62d46" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:57.878455 10541 raft_consensus.cc:2468] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 0.
I20260709 13:13:57.878801 10592 raft_consensus.cc:2804] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.880517 10592 raft_consensus.cc:697] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:13:57.881368 10592 consensus_queue.cc:237] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.883309 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:57.882949 10392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597045aaee954382839393fe2ac62d46" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:57.883333 10391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:57.887869 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
W20260709 13:13:57.886638 10433 leader_election.cc:343] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.889875 10391 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 0.
W20260709 13:13:57.892887 10433 leader_election.cc:343] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.893563 10433 leader_election.cc:304] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c412495aa9434b800110f744218c0a, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:57.896370 10608 raft_consensus.cc:2804] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.896978 10608 raft_consensus.cc:493] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.896938 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:57.897830 10541 raft_consensus.cc:2468] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 0.
I20260709 13:13:57.898229 10432 leader_election.cc:304] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: cd0332521e0b4db09972c056204d24e3
I20260709 13:13:57.899045 10590 raft_consensus.cc:2804] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.897416 10608 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.899389 10590 raft_consensus.cc:493] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.899919 10590 raft_consensus.cc:3060] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:57.900378 10432 leader_election.cc:343] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.904814 10432 leader_election.cc:304] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: cd0332521e0b4db09972c056204d24e3
I20260709 13:13:57.905699 10467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:57.906136 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
W20260709 13:13:57.910008 10356 leader_election.cc:343] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6c412495aa9434b800110f744218c0a (127.10.4.66:42071): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.910454 10584 tablet_bootstrap.cc:654] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.914054 10608 raft_consensus.cc:515] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.920749 10467 raft_consensus.cc:2393] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd0332521e0b4db09972c056204d24e3 in current term 1: Already voted for candidate c6c412495aa9434b800110f744218c0a in this term.
I20260709 13:13:57.922817 10590 raft_consensus.cc:515] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.924973 10590 leader_election.cc:290] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:57.930228 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597045aaee954382839393fe2ac62d46" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:57.928922 10391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597045aaee954382839393fe2ac62d46" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:57.928918 10616 raft_consensus.cc:2804] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.931725 10541 raft_consensus.cc:3060] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.932711 10608 leader_election.cc:290] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:57.932634 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
W20260709 13:13:57.933020 10433 leader_election.cc:343] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.932006 10616 raft_consensus.cc:493] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.934374 10390 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.935369 10586 tablet_bootstrap.cc:492] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:57.935432 10616 raft_consensus.cc:3060] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.937539 10586 ts_tablet_manager.cc:1403] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.014s
I20260709 13:13:57.939251 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:57.939710 10585 tablet_bootstrap.cc:492] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:57.940500 10585 ts_tablet_manager.cc:1403] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.166s	user 0.007s	sys 0.011s
W20260709 13:13:57.943463 10432 leader_election.cc:343] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.946192 10584 tablet_bootstrap.cc:492] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:57.945926 10585 raft_consensus.cc:359] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.946650 10584 ts_tablet_manager.cc:1403] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.003s
I20260709 13:13:57.946671 10585 raft_consensus.cc:385] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.947211 10585 raft_consensus.cc:740] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.946595 10586 raft_consensus.cc:359] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.948223 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:57.948843 10540 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 0.
I20260709 13:13:57.948382 10585 consensus_queue.cc:260] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.948378 10586 raft_consensus.cc:385] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.949609 10586 raft_consensus.cc:740] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.949824 10541 raft_consensus.cc:2468] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 1.
I20260709 13:13:57.950237 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:57.950529 10586 consensus_queue.cc:260] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.950467 10584 raft_consensus.cc:359] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.950851 10540 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 0.
I20260709 13:13:57.951972 10584 raft_consensus.cc:385] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:57.952740 10584 raft_consensus.cc:740] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:57.952957 10357 leader_election.cc:304] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: c6c412495aa9434b800110f744218c0a
I20260709 13:13:57.954083 10390 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 1.
I20260709 13:13:57.954963 10612 raft_consensus.cc:2804] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:57.953118 10432 leader_election.cc:304] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: cd0332521e0b4db09972c056204d24e3
I20260709 13:13:57.955587 10357 leader_election.cc:304] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:57.956423 10612 raft_consensus.cc:493] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.956743 10591 raft_consensus.cc:2764] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:13:57.957311 10608 raft_consensus.cc:2804] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.956804 10612 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.957772 10433 leader_election.cc:304] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c6c412495aa9434b800110f744218c0a, cd0332521e0b4db09972c056204d24e3; no voters: 98ef1ab89307452d8cd1dfa48d30564c
I20260709 13:13:57.957840 10585 ts_tablet_manager.cc:1434] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:13:57.959196 10586 ts_tablet_manager.cc:1434] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:13:57.953902 10584 consensus_queue.cc:260] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.957803 10608 raft_consensus.cc:697] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:57.959637 10590 raft_consensus.cc:2804] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:57.961063 10586 tablet_bootstrap.cc:492] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:57.960950 10608 consensus_queue.cc:237] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.962087 10584 ts_tablet_manager.cc:1434] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:13:57.966176 10585 tablet_bootstrap.cc:492] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:57.967370 10584 tablet_bootstrap.cc:492] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:57.979117 10612 raft_consensus.cc:515] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:57.978399 10590 raft_consensus.cc:697] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:57.981379 10590 consensus_queue.cc:237] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:57.984285 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a"
W20260709 13:13:57.986008 10356 leader_election.cc:343] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c6c412495aa9434b800110f744218c0a (127.10.4.66:42071): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:57.982575 10612 leader_election.cc:290] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 election: Requested vote from peers 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:57.987696 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:57.988607 10540 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:57.995117 10585 tablet_bootstrap.cc:654] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.995455 10586 tablet_bootstrap.cc:654] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:57.998689 10616 raft_consensus.cc:515] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.002995 10616 leader_election.cc:290] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.011823 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:58.004760 10540 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 1.
I20260709 13:13:58.013767 10541 raft_consensus.cc:3060] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.015154 10592 raft_consensus.cc:493] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.015755 10592 raft_consensus.cc:515] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.018049 10592 leader_election.cc:290] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:58.019375 10357 leader_election.cc:304] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.020255 10612 raft_consensus.cc:2804] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.020735 10612 raft_consensus.cc:697] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:13:58.021584 10391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:58.021502 10612 consensus_queue.cc:237] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:58.022222 10391 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 0.
I20260709 13:13:58.023608 10507 leader_election.cc:304] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:58.024524 10592 raft_consensus.cc:2804] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.024859 10592 raft_consensus.cc:493] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.025202 10592 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.027609 10287 catalog_manager.cc:5697] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 1 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.005129 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:58.031739 10467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
W20260709 13:13:58.032842 10433 leader_election.cc:343] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.037590 10592 raft_consensus.cc:515] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
W20260709 13:13:58.033238 10505 leader_election.cc:343] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c6c412495aa9434b800110f744218c0a (127.10.4.66:42071): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.040714 10541 raft_consensus.cc:2468] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 1.
I20260709 13:13:58.042611 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:58.043190 10390 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.001545 10584 tablet_bootstrap.cc:654] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.050117 10390 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 1.
I20260709 13:13:58.051596 10507 leader_election.cc:304] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:58.052654 10604 raft_consensus.cc:2804] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.060384 10592 leader_election.cc:290] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:58.061717 10467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:13:58.062767 10432 leader_election.cc:304] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: cd0332521e0b4db09972c056204d24e3
W20260709 13:13:58.063189 10505 leader_election.cc:343] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c6c412495aa9434b800110f744218c0a (127.10.4.66:42071): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.063695 10590 raft_consensus.cc:2804] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.064167 10590 raft_consensus.cc:697] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:58.065006 10590 consensus_queue.cc:237] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.072125 10584 tablet_bootstrap.cc:492] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:58.072638 10585 tablet_bootstrap.cc:492] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:58.075021 10586 tablet_bootstrap.cc:492] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:58.076932 10586 ts_tablet_manager.cc:1403] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.116s	user 0.020s	sys 0.011s
I20260709 13:13:58.077508 10584 ts_tablet_manager.cc:1403] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.111s	user 0.007s	sys 0.014s
I20260709 13:13:58.076930 10585 ts_tablet_manager.cc:1403] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.112s	user 0.006s	sys 0.020s
I20260709 13:13:58.079905 10604 raft_consensus.cc:697] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:13:58.083904 10604 consensus_queue.cc:237] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.085878 10585 raft_consensus.cc:359] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.087117 10585 raft_consensus.cc:385] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.087446 10585 raft_consensus.cc:740] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.088214 10585 consensus_queue.cc:260] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.090677 10586 raft_consensus.cc:359] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.091511 10586 raft_consensus.cc:385] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.091814 10586 raft_consensus.cc:740] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.092761 10586 consensus_queue.cc:260] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.095814 10289 catalog_manager.cc:5697] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 0 to 1, leader changed from <none> to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 1 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.096920 10585 ts_tablet_manager.cc:1434] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 13:13:58.098155 10585 tablet_bootstrap.cc:492] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:58.097817 10289 catalog_manager.cc:5697] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 0 to 1, leader changed from <none> to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 1 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.106499 10586 ts_tablet_manager.cc:1434] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:13:58.107338 10586 tablet_bootstrap.cc:492] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:58.110914 10286 catalog_manager.cc:5697] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.116341 10584 raft_consensus.cc:359] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.117126 10584 raft_consensus.cc:385] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.117444 10584 raft_consensus.cc:740] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.117856 10586 tablet_bootstrap.cc:654] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.118600 10585 tablet_bootstrap.cc:654] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.118321 10584 consensus_queue.cc:260] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.123875 10584 ts_tablet_manager.cc:1434] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.003s
I20260709 13:13:58.124907 10584 tablet_bootstrap.cc:492] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:58.124131 10287 catalog_manager.cc:5697] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 1 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.133662 10584 tablet_bootstrap.cc:654] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.143151 10586 tablet_bootstrap.cc:492] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:58.143620 10586 ts_tablet_manager.cc:1403] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.004s
I20260709 13:13:58.146366 10586 raft_consensus.cc:359] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:58.147053 10586 raft_consensus.cc:385] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.147334 10586 raft_consensus.cc:740] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.148084 10586 consensus_queue.cc:260] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:58.148409 10286 catalog_manager.cc:5697] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 0 to 1, leader changed from <none> to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 1 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.150372 10586 ts_tablet_manager.cc:1434] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:13:58.151381 10586 tablet_bootstrap.cc:492] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:58.152701 10585 tablet_bootstrap.cc:492] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:58.153178 10585 ts_tablet_manager.cc:1403] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.055s	user 0.006s	sys 0.009s
I20260709 13:13:58.156327 10585 raft_consensus.cc:359] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.159667 10585 raft_consensus.cc:385] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.160001 10585 raft_consensus.cc:740] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.160782 10585 consensus_queue.cc:260] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.163120 10585 ts_tablet_manager.cc:1434] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:13:58.164135 10585 tablet_bootstrap.cc:492] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c: Bootstrap starting.
I20260709 13:13:58.164077 10590 raft_consensus.cc:493] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.164633 10590 raft_consensus.cc:515] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.165970 10616 raft_consensus.cc:493] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.166381 10586 tablet_bootstrap.cc:654] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.166761 10590 leader_election.cc:290] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.168292 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.169730 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:58.170272 10584 tablet_bootstrap.cc:492] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:58.170742 10584 ts_tablet_manager.cc:1403] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.018s
I20260709 13:13:58.171545 10390 raft_consensus.cc:2393] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c6c412495aa9434b800110f744218c0a in current term 1: Already voted for candidate 98ef1ab89307452d8cd1dfa48d30564c in this term.
I20260709 13:13:58.172783 10433 leader_election.cc:304] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6c412495aa9434b800110f744218c0a; no voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.173645 10585 tablet_bootstrap.cc:654] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.173692 10584 raft_consensus.cc:359] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.166424 10616 raft_consensus.cc:515] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.174479 10584 raft_consensus.cc:385] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.174805 10584 raft_consensus.cc:740] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.175999 10616 leader_election.cc:290] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.178514 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.179113 10541 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 0.
I20260709 13:13:58.180581 10432 leader_election.cc:304] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: 
I20260709 13:13:58.181635 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:58.182386 10616 raft_consensus.cc:2804] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.182861 10616 raft_consensus.cc:493] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.183171 10616 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.173861 10590 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.189702 10586 tablet_bootstrap.cc:492] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:58.190205 10586 ts_tablet_manager.cc:1403] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.039s	user 0.006s	sys 0.011s
I20260709 13:13:58.192992 10586 raft_consensus.cc:359] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.193710 10586 raft_consensus.cc:385] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.193980 10586 raft_consensus.cc:740] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.194468 10590 raft_consensus.cc:2749] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.195993 10590 raft_consensus.cc:493] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.196491 10590 raft_consensus.cc:515] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.175626 10584 consensus_queue.cc:260] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.211266 10390 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 0.
I20260709 13:13:58.212383 10584 ts_tablet_manager.cc:1434] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.004s
I20260709 13:13:58.212983 10612 raft_consensus.cc:493] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.213478 10584 tablet_bootstrap.cc:492] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:58.213990 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.213542 10612 raft_consensus.cc:515] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.194890 10586 consensus_queue.cc:260] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.207322 10391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:58.208738 10608 raft_consensus.cc:493] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.208323 10616 raft_consensus.cc:515] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.225234 10391 raft_consensus.cc:2393] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c6c412495aa9434b800110f744218c0a in current term 1: Already voted for candidate 98ef1ab89307452d8cd1dfa48d30564c in this term.
I20260709 13:13:58.226595 10433 leader_election.cc:304] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6c412495aa9434b800110f744218c0a; no voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.228125 10618 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.210003 10590 leader_election.cc:290] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.224311 10591 raft_consensus.cc:493] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.224227 10608 raft_consensus.cc:515] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:13:58.230471 10591 raft_consensus.cc:515] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.234236 10608 leader_election.cc:290] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.237488 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:58.239559 10467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:58.239908 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:58.240337 10467 raft_consensus.cc:2393] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd0332521e0b4db09972c056204d24e3 in current term 1: Already voted for candidate c6c412495aa9434b800110f744218c0a in this term.
I20260709 13:13:58.239187 10612 leader_election.cc:290] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.243013 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.243085 10616 leader_election.cc:290] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.243741 10541 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 0.
I20260709 13:13:58.245329 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.245406 10357 leader_election.cc:304] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.245760 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:58.247090 10612 raft_consensus.cc:2804] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.247481 10539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:58.248355 10539 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.247766 10390 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.250059 10586 ts_tablet_manager.cc:1434] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.060s	user 0.012s	sys 0.000s
I20260709 13:13:58.249001 10540 raft_consensus.cc:2393] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c6c412495aa9434b800110f744218c0a in current term 1: Already voted for candidate cd0332521e0b4db09972c056204d24e3 in this term.
I20260709 13:13:58.250937 10586 tablet_bootstrap.cc:492] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a: Bootstrap starting.
I20260709 13:13:58.251654 10432 leader_election.cc:304] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6c412495aa9434b800110f744218c0a; no voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.254474 10591 leader_election.cc:290] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.255024 10584 tablet_bootstrap.cc:654] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.256155 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.257340 10540 raft_consensus.cc:2393] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd0332521e0b4db09972c056204d24e3 in current term 1: Already voted for candidate c6c412495aa9434b800110f744218c0a in this term.
I20260709 13:13:58.258520 10357 leader_election.cc:304] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd0332521e0b4db09972c056204d24e3; no voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.259379 10390 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 1.
I20260709 13:13:58.259932 10612 raft_consensus.cc:493] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.260298 10612 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.260706 10539 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 1.
I20260709 13:13:58.261094 10433 leader_election.cc:304] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c412495aa9434b800110f744218c0a, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:58.261929 10616 raft_consensus.cc:2804] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.262462 10616 raft_consensus.cc:697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:58.263175 10636 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.259537 10591 raft_consensus.cc:3060] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.263314 10616 consensus_queue.cc:237] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.268985 10612 raft_consensus.cc:515] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.270644 10591 raft_consensus.cc:2749] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.276052 10591 raft_consensus.cc:493] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.270951 10618 raft_consensus.cc:2749] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.276976 10585 tablet_bootstrap.cc:492] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c: No bootstrap required, opened a new log
I20260709 13:13:58.277407 10585 ts_tablet_manager.cc:1403] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent bootstrapping tablet: real 0.113s	user 0.016s	sys 0.057s
I20260709 13:13:58.276903 10591 raft_consensus.cc:515] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.279119 10591 leader_election.cc:290] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.279197 10612 leader_election.cc:290] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Requested vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.280123 10584 tablet_bootstrap.cc:492] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:58.280542 10584 ts_tablet_manager.cc:1403] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.000s
I20260709 13:13:58.283344 10586 tablet_bootstrap.cc:654] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.284065 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:13:58.284782 10467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597045aaee954382839393fe2ac62d46" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:58.287336 10636 raft_consensus.cc:2749] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.285421 10585 raft_consensus.cc:359] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.288441 10585 raft_consensus.cc:385] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.288760 10585 raft_consensus.cc:740] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.289845 10584 raft_consensus.cc:359] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.290481 10539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597045aaee954382839393fe2ac62d46" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.291193 10539 raft_consensus.cc:2393] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd0332521e0b4db09972c056204d24e3 in current term 1: Already voted for candidate c6c412495aa9434b800110f744218c0a in this term.
I20260709 13:13:58.291141 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:58.291844 10540 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.290455 10584 raft_consensus.cc:385] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.293166 10584 raft_consensus.cc:740] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.292562 10585 consensus_queue.cc:260] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.293772 10584 consensus_queue.cc:260] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.295723 10585 ts_tablet_manager.cc:1434] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:13:58.298430 10288 catalog_manager.cc:5697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 0 to 1, leader changed from <none> to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 1 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.302714 10540 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:13:58.303889 10586 tablet_bootstrap.cc:492] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a: No bootstrap required, opened a new log
I20260709 13:13:58.304349 10586 ts_tablet_manager.cc:1403] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.003s
I20260709 13:13:58.307361 10586 raft_consensus.cc:359] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.308136 10586 raft_consensus.cc:385] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.308439 10586 raft_consensus.cc:740] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.309434 10584 ts_tablet_manager.cc:1434] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 13:13:58.309194 10586 consensus_queue.cc:260] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.311254 10586 ts_tablet_manager.cc:1434] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:13:58.312649 10584 tablet_bootstrap.cc:492] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3: Bootstrap starting.
I20260709 13:13:58.313723 10626 raft_consensus.cc:493] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.314342 10626 raft_consensus.cc:515] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.317200 10584 tablet_bootstrap.cc:654] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:58.318518 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1340349398384547b61df98ecaf81f90" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
I20260709 13:13:58.319195 10390 raft_consensus.cc:2468] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 0.
I20260709 13:13:58.319444 10357 leader_election.cc:304] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd0332521e0b4db09972c056204d24e3; no voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.320639 10638 raft_consensus.cc:3060] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.321084 10507 leader_election.cc:304] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:58.321902 10604 raft_consensus.cc:2804] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.322254 10604 raft_consensus.cc:493] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.322574 10604 raft_consensus.cc:3060] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.327802 10467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1340349398384547b61df98ecaf81f90" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:58.328296 10638 raft_consensus.cc:2749] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:58.328454 10467 raft_consensus.cc:2468] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 0.
I20260709 13:13:58.329840 10357 leader_election.cc:304] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.330873 10626 leader_election.cc:290] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:58.331291 10638 raft_consensus.cc:2804] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:58.331918 10638 raft_consensus.cc:697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:13:58.332815 10638 consensus_queue.cc:237] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.338070 10604 raft_consensus.cc:515] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.342892 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1340349398384547b61df98ecaf81f90" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:58.343478 10390 raft_consensus.cc:3060] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.351995 10626 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:13:58.365491 10390 raft_consensus.cc:2468] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 1.
I20260709 13:13:58.367585 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1340349398384547b61df98ecaf81f90" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:13:58.397723 10466 raft_consensus.cc:3060] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.379438 10540 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.409759 10646 raft_consensus.cc:493] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.410640 10646 raft_consensus.cc:515] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.412801 10646 leader_election.cc:290] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:58.409893 10466 raft_consensus.cc:2468] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 1.
I20260709 13:13:58.414186 10616 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.414597 10391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3" is_pre_election: true
W20260709 13:13:58.416437 10507 leader_election.cc:343] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.417137 10507 leader_election.cc:304] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:58.418028 10646 raft_consensus.cc:2804] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.418543 10646 raft_consensus.cc:697] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:13:58.419543 10646 consensus_queue.cc:237] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.421790 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:58.422470 10466 raft_consensus.cc:2468] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 0.
I20260709 13:13:58.423707 10505 leader_election.cc:304] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.396742 10604 leader_election.cc:290] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:58.392071 10626 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:58.424887 10647 raft_consensus.cc:2804] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:58.425247 10647 raft_consensus.cc:493] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.425546 10647 raft_consensus.cc:3060] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.432662 10647 raft_consensus.cc:515] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.435905 10647 leader_election.cc:290] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:13:58.436055 10391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0332521e0b4db09972c056204d24e3"
W20260709 13:13:58.439201 10507 leader_election.cc:343] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:58.440719 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:13:58.441358 10466 raft_consensus.cc:3060] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.457566 10616 consensus_queue.cc:1048] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:13:58.465659 10289 catalog_manager.cc:5697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 1 to 2, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.467747 10286 catalog_manager.cc:5697] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 1 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.473737 10616 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:58.480082 10584 tablet_bootstrap.cc:492] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3: No bootstrap required, opened a new log
I20260709 13:13:58.480602 10584 ts_tablet_manager.cc:1403] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3: Time spent bootstrapping tablet: real 0.168s	user 0.048s	sys 0.083s
I20260709 13:13:58.486874 10618 consensus_queue.cc:1048] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:13:58.488317 10584 raft_consensus.cc:359] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.488941 10584 raft_consensus.cc:385] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:58.489198 10584 raft_consensus.cc:740] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Initialized, Role: FOLLOWER
I20260709 13:13:58.489899 10584 consensus_queue.cc:260] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.496394 10584 ts_tablet_manager.cc:1434] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.005s
I20260709 13:13:58.500267 10466 raft_consensus.cc:2468] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 1.
I20260709 13:13:58.501744 10505 leader_election.cc:304] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.503118 10646 raft_consensus.cc:2804] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:58.503687 10646 raft_consensus.cc:697] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:13:58.504638 10646 consensus_queue.cc:237] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.518874 10616 consensus_queue.cc:1048] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:13:58.524616 10337 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:58.527688 10638 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.533879 10616 consensus_queue.cc:1048] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.542385 10286 catalog_manager.cc:5697] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 1 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.558408 10591 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.603055 10647 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:58.613994 10604 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.711315 10541 raft_consensus.cc:1240] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Rejecting Update request from peer c6c412495aa9434b800110f744218c0a for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:58.713035 10590 consensus_queue.cc:1059] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.714036 10608 raft_consensus.cc:3055] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:58.714325 10608 raft_consensus.cc:740] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:58.715174 10608 consensus_queue.cc:260] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.716158 10608 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.747617 10541 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:13:58.748803 10591 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.759151 10465 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:13:58.759773 10465 pending_rounds.cc:85] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: Aborting all ops after (but not including) 0
I20260709 13:13:58.760097 10465 pending_rounds.cc:107] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:13:58.760375 10465 raft_consensus.cc:2889] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:58.767084 10638 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.788218 10541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
mode: GRACEFUL
new_leader_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:43316
I20260709 13:13:58.788808 10541 raft_consensus.cc:606] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Received request to transfer leadership to TS cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.794390 10541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73845cf648b3497c9309033a7379b5be"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
mode: GRACEFUL
new_leader_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:43316
I20260709 13:13:58.794904 10541 raft_consensus.cc:606] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Received request to transfer leadership to TS cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.809645 10465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
mode: GRACEFUL
new_leader_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:51504
I20260709 13:13:58.810178 10465 raft_consensus.cc:606] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Received request to transfer leadership to TS cd0332521e0b4db09972c056204d24e3
I20260709 13:13:58.813321 10604 consensus_queue.cc:1048] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.830120 10647 consensus_queue.cc:1048] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:58.835500 10391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
mode: GRACEFUL
new_leader_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:37264
I20260709 13:13:58.836139 10391 raft_consensus.cc:606] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Received request to transfer leadership to TS c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.842517 10541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
mode: GRACEFUL
new_leader_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:43316
I20260709 13:13:58.843118 10541 raft_consensus.cc:606] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Received request to transfer leadership to TS c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.887717 10647 raft_consensus.cc:993] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c: : Instructing follower cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:13:58.888136 10647 raft_consensus.cc:1081] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Signalling peer cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:13:58.889650 10392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:37234
I20260709 13:13:58.890108 10392 raft_consensus.cc:493] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:58.890389 10392 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.895974 10392 raft_consensus.cc:515] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.897864 10392 leader_election.cc:290] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Requested vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.898466 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:13:58.898941 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:58.899114 10465 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.899446 10540 raft_consensus.cc:3055] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:58.899746 10540 raft_consensus.cc:740] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:13:58.900420 10540 consensus_queue.cc:260] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.901273 10540 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.905370 10540 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:13:58.905818 10465 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:13:58.906451 10357 leader_election.cc:304] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:58.907194 10638 raft_consensus.cc:2804] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:58.907691 10638 raft_consensus.cc:697] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:13:58.908486 10638 consensus_queue.cc:237] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.915769 10286 catalog_manager.cc:5697] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 1 to 2, leader changed from 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67) to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:58.952678 10638 raft_consensus.cc:493] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.953187 10638 raft_consensus.cc:515] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:58.954999 10638 leader_election.cc:290] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:58.957759 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c" is_pre_election: true
I20260709 13:13:58.957259 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c412495aa9434b800110f744218c0a" is_pre_election: true
I20260709 13:13:58.959964 10465 raft_consensus.cc:2393] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd0332521e0b4db09972c056204d24e3 in current term 1: Already voted for candidate 98ef1ab89307452d8cd1dfa48d30564c in this term.
I20260709 13:13:58.960878 10356 leader_election.cc:304] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd0332521e0b4db09972c056204d24e3; no voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a
I20260709 13:13:58.961915 10638 raft_consensus.cc:3060] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:58.966305 10638 raft_consensus.cc:2749] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:59.063236 10647 consensus_queue.cc:1048] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.072813 10646 consensus_queue.cc:1048] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:59.082218 10646 raft_consensus.cc:993] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c: : Instructing follower c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:13:59.082659 10646 raft_consensus.cc:1081] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Signalling peer c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:13:59.084481 10465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:51492
I20260709 13:13:59.085043 10465 raft_consensus.cc:493] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:59.085397 10465 raft_consensus.cc:3060] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.090440 10465 raft_consensus.cc:515] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.091431 10590 raft_consensus.cc:993] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: : Instructing follower cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:13:59.091857 10590 raft_consensus.cc:1081] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Signalling peer cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:13:59.092804 10465 leader_election.cc:290] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 2 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:59.095554 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:59.096113 10540 raft_consensus.cc:3055] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:59.096413 10540 raft_consensus.cc:740] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:13:59.097169 10540 consensus_queue.cc:260] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.098353 10540 raft_consensus.cc:3060] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.098793 10391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:37250
I20260709 13:13:59.099107 10392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a54d74c40c043b8a6daab30bc33d4fd" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:59.099064 10604 raft_consensus.cc:993] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: : Instructing follower cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:13:59.099501 10391 raft_consensus.cc:493] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:59.099757 10392 raft_consensus.cc:3060] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.099992 10391 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.100242 10626 raft_consensus.cc:1081] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Signalling peer cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:13:59.101918 10390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73845cf648b3497c9309033a7379b5be"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:37234
I20260709 13:13:59.102417 10390 raft_consensus.cc:493] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:59.102758 10390 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.106421 10391 raft_consensus.cc:515] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.108750 10390 raft_consensus.cc:515] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.111567 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:13:59.112248 10466 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.112440 10465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:13:59.112995 10465 raft_consensus.cc:3055] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:59.113214 10540 raft_consensus.cc:2468] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 2.
I20260709 13:13:59.114197 10391 leader_election.cc:290] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Requested vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:59.113313 10392 raft_consensus.cc:2468] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 2.
I20260709 13:13:59.113312 10465 raft_consensus.cc:740] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:59.115895 10432 leader_election.cc:304] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: 
I20260709 13:13:59.117063 10616 raft_consensus.cc:2804] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:59.116956 10465 consensus_queue.cc:260] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.117522 10616 raft_consensus.cc:697] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 2 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:59.118005 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:59.118314 10616 consensus_queue.cc:237] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.119091 10540 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.120276 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:59.120838 10541 raft_consensus.cc:3055] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:59.121171 10541 raft_consensus.cc:740] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:13:59.121922 10541 consensus_queue.cc:260] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.123006 10541 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.124881 10466 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:13:59.126012 10540 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:13:59.126125 10356 leader_election.cc:304] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c412495aa9434b800110f744218c0a, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:59.126925 10638 raft_consensus.cc:2804] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:59.128106 10357 leader_election.cc:304] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:13:59.128466 10638 raft_consensus.cc:697] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:13:59.118198 10465 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:59.118600 10390 leader_election.cc:290] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Requested vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:59.129280 10541 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:13:59.129801 10591 raft_consensus.cc:2804] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:59.130316 10591 raft_consensus.cc:697] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:13:59.131189 10591 consensus_queue.cc:237] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.129336 10638 consensus_queue.cc:237] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.138473 10465 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:13:59.140667 10289 catalog_manager.cc:5697] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 1 to 2, leader changed from 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67) to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.149447 10286 catalog_manager.cc:5697] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 1 to 2, leader changed from 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67) to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 2 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.154901 10289 catalog_manager.cc:5697] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 1 to 2, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.299857 10591 raft_consensus.cc:993] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3: : Instructing follower c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:13:59.300311 10638 raft_consensus.cc:1081] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Signalling peer c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:13:59.301951 10466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:51496
I20260709 13:13:59.302484 10466 raft_consensus.cc:493] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:59.302832 10466 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.310668 10466 raft_consensus.cc:515] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.312489 10466 leader_election.cc:290] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:13:59.313515 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:13:59.313619 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:13:59.313968 10541 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.314118 10390 raft_consensus.cc:3055] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:59.314414 10390 raft_consensus.cc:740] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:13:59.315038 10390 consensus_queue.cc:260] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.315927 10390 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.318485 10541 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:13:59.319489 10432 leader_election.cc:304] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: 
I20260709 13:13:59.320173 10616 raft_consensus.cc:2804] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:59.320482 10390 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:13:59.320581 10616 raft_consensus.cc:697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:13:59.321350 10616 consensus_queue.cc:237] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:13:59.327064 10289 catalog_manager.cc:5697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 2 to 3, leader changed from cd0332521e0b4db09972c056204d24e3 (127.10.4.65) to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 3 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.354115 10466 raft_consensus.cc:1275] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.355446 10638 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.362674 10541 raft_consensus.cc:1275] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.364238 10612 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.513756 10541 raft_consensus.cc:1275] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.515204 10612 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.522352 10466 raft_consensus.cc:1275] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.523934 10638 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.531227 10541 raft_consensus.cc:1275] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.532783 10638 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.548199 10466 raft_consensus.cc:1275] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.549688 10612 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.550254 10390 raft_consensus.cc:1275] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.551553 10616 consensus_queue.cc:1048] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.560947 10541 raft_consensus.cc:1275] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:59.562352 10616 consensus_queue.cc:1048] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.817056 10390 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:59.818164 10616 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.827245 10541 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:59.828742 10608 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:01.878073 10257 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:14:01.878825 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:01.900223 10390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
mode: GRACEFUL
new_leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:52370
I20260709 13:14:01.900702 10390 raft_consensus.cc:606] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 LEADER]: Received request to transfer leadership to TS 98ef1ab89307452d8cd1dfa48d30564c
I20260709 13:14:01.910353 10466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
mode: GRACEFUL
new_leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:50726
I20260709 13:14:01.910859 10466 raft_consensus.cc:606] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Received request to transfer leadership to TS 98ef1ab89307452d8cd1dfa48d30564c
I20260709 13:14:01.912055 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:01.912470 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:02.038409 10608 raft_consensus.cc:993] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: : Instructing follower 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:02.038898 10676 raft_consensus.cc:1081] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Signalling peer 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:02.040488 10541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:43290
I20260709 13:14:02.041035 10541 raft_consensus.cc:493] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:02.041375 10541 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:02.045814 10541 raft_consensus.cc:515] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:02.047863 10541 leader_election.cc:290] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 4 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:14:02.048368 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:14:02.048432 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:02.048928 10466 raft_consensus.cc:3055] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:02.049031 10390 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:02.049247 10466 raft_consensus.cc:740] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:02.049983 10466 consensus_queue.cc:260] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:02.050848 10466 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:02.053864 10390 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 4.
I20260709 13:14:02.054996 10507 leader_election.cc:304] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:02.055540 10466 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 4.
I20260709 13:14:02.055670 10626 raft_consensus.cc:2804] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:02.056869 10626 raft_consensus.cc:697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:14:02.057582 10626 consensus_queue.cc:237] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:02.064096 10289 catalog_manager.cc:5697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 3 to 4, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 4 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:02.181707 10591 raft_consensus.cc:993] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3: : Instructing follower 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:02.182085 10686 raft_consensus.cc:1081] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 LEADER]: Signalling peer 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:02.183557 10541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:43300
I20260709 13:14:02.183969 10541 raft_consensus.cc:493] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:02.184233 10541 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:02.188481 10541 raft_consensus.cc:515] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:02.190021 10541 leader_election.cc:290] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 2 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:14:02.190964 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:02.191116 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:14:02.191771 10466 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:02.191753 10390 raft_consensus.cc:3055] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:02.192313 10390 raft_consensus.cc:740] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:14:02.193126 10390 consensus_queue.cc:260] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:02.194352 10390 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:02.198834 10466 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 2.
I20260709 13:14:02.200066 10505 leader_election.cc:304] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: 
I20260709 13:14:02.200754 10390 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 2.
I20260709 13:14:02.200896 10626 raft_consensus.cc:2804] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:02.201418 10626 raft_consensus.cc:697] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:14:02.202240 10626 consensus_queue.cc:237] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:02.212306 10289 catalog_manager.cc:5697] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 1 to 2, leader changed from cd0332521e0b4db09972c056204d24e3 (127.10.4.65) to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 2 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:02.592455 10466 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:02.593652 10626 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:02.602089 10390 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:02.603466 10690 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:02.728086 10390 raft_consensus.cc:1275] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:02.729305 10690 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:02.738322 10466 raft_consensus.cc:1275] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:02.739609 10690 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:02.913236 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:02.922845 10541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
mode: GRACEFUL
new_leader_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:43316
I20260709 13:14:02.923341 10541 raft_consensus.cc:606] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 LEADER]: Received request to transfer leadership to TS c6c412495aa9434b800110f744218c0a
I20260709 13:14:02.928138 10390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
mode: GRACEFUL
new_leader_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:37264
I20260709 13:14:02.928634 10390 raft_consensus.cc:606] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Received request to transfer leadership to TS c6c412495aa9434b800110f744218c0a
I20260709 13:14:02.933642 10390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73845cf648b3497c9309033a7379b5be"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
mode: GRACEFUL
new_leader_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:37264
I20260709 13:14:02.934151 10390 raft_consensus.cc:606] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Received request to transfer leadership to TS c6c412495aa9434b800110f744218c0a
I20260709 13:14:02.938637 10390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
mode: GRACEFUL
new_leader_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:37264
I20260709 13:14:02.939144 10390 raft_consensus.cc:606] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Received request to transfer leadership to TS c6c412495aa9434b800110f744218c0a
I20260709 13:14:02.949880 10686 raft_consensus.cc:993] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3: : Instructing follower c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:02.950246 10541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
mode: GRACEFUL
new_leader_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:43316
I20260709 13:14:02.950392 10694 raft_consensus.cc:1081] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Signalling peer c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:02.950847 10541 raft_consensus.cc:606] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Received request to transfer leadership to TS c6c412495aa9434b800110f744218c0a
I20260709 13:14:02.952219 10466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:51496
I20260709 13:14:02.952769 10466 raft_consensus.cc:493] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:02.953122 10466 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:02.957995 10466 raft_consensus.cc:515] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:02.959753 10466 leader_election.cc:290] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:14:02.960474 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:02.960707 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:02.961138 10390 raft_consensus.cc:3055] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:02.961151 10541 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:02.961526 10390 raft_consensus.cc:740] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:14:02.962218 10390 consensus_queue.cc:260] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:02.963099 10390 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:02.966102 10541 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:02.967171 10432 leader_election.cc:304] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: 
I20260709 13:14:02.967905 10676 raft_consensus.cc:2804] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:02.968977 10676 raft_consensus.cc:697] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:02.969275 10390 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:02.969818 10676 consensus_queue.cc:237] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:02.977196 10289 catalog_manager.cc:5697] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 2 to 3, leader changed from cd0332521e0b4db09972c056204d24e3 (127.10.4.65) to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 3 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:03.042289 10626 raft_consensus.cc:993] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c: : Instructing follower c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.042703 10690 raft_consensus.cc:1081] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Signalling peer c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.044359 10466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:51492
I20260709 13:14:03.044868 10466 raft_consensus.cc:493] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:03.045151 10466 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:03.049739 10466 raft_consensus.cc:515] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.051331 10466 leader_election.cc:290] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 5 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:14:03.052296 10392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:03.052385 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3836d83f3cc483b8efb7ebd8e67dfd1" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:03.052953 10392 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:03.053030 10541 raft_consensus.cc:3055] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:03.053421 10541 raft_consensus.cc:740] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:14:03.053984 10541 consensus_queue.cc:260] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.054926 10541 raft_consensus.cc:3060] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:03.057873 10392 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 5.
I20260709 13:14:03.058983 10541 raft_consensus.cc:2468] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 5.
I20260709 13:14:03.059177 10433 leader_election.cc:304] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c412495aa9434b800110f744218c0a, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:03.060030 10676 raft_consensus.cc:2804] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:03.060495 10676 raft_consensus.cc:697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 5 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:03.061348 10676 consensus_queue.cc:237] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.066735 10289 catalog_manager.cc:5697] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 4 to 5, leader changed from 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67) to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 5 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:03.124187 10686 raft_consensus.cc:993] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3: : Instructing follower c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.124564 10694 raft_consensus.cc:1081] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Signalling peer c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.125841 10466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:51496
I20260709 13:14:03.126262 10466 raft_consensus.cc:493] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:03.126506 10466 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.130978 10466 raft_consensus.cc:515] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.132457 10466 leader_election.cc:290] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:14:03.133183 10392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:03.133416 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:03.133687 10392 raft_consensus.cc:3055] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:03.133865 10541 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.133982 10392 raft_consensus.cc:740] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:14:03.134667 10392 consensus_queue.cc:260] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.135548 10392 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.137691 10541 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:03.138528 10432 leader_election.cc:304] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, c6c412495aa9434b800110f744218c0a; no voters: 
I20260709 13:14:03.139129 10676 raft_consensus.cc:2804] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:03.139605 10676 raft_consensus.cc:697] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:03.140318 10392 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:03.140367 10676 consensus_queue.cc:237] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.146642 10289 catalog_manager.cc:5697] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 2 to 3, leader changed from cd0332521e0b4db09972c056204d24e3 (127.10.4.65) to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 3 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:03.170250 10626 raft_consensus.cc:993] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c: : Instructing follower c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.170728 10690 raft_consensus.cc:1081] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 LEADER]: Signalling peer c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.172359 10466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:51492
I20260709 13:14:03.173202 10466 raft_consensus.cc:493] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:03.173579 10466 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.178066 10466 raft_consensus.cc:515] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:03.179937 10466 leader_election.cc:290] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:14:03.180828 10392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:03.181018 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:03.181368 10392 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.181730 10541 raft_consensus.cc:3055] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:03.182050 10541 raft_consensus.cc:740] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:14:03.182750 10541 consensus_queue.cc:260] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:03.183851 10541 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.187608 10392 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:03.188707 10541 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:03.188896 10433 leader_election.cc:304] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c412495aa9434b800110f744218c0a, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:03.189800 10676 raft_consensus.cc:2804] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:03.190320 10676 raft_consensus.cc:697] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:03.191133 10676 consensus_queue.cc:237] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:03.197531 10289 catalog_manager.cc:5697] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 2 to 3, leader changed from 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67) to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 3 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.363039 10392 raft_consensus.cc:1275] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.364310 10676 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:03.367417 10694 raft_consensus.cc:993] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3: : Instructing follower c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.367916 10686 raft_consensus.cc:1081] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Signalling peer c6c412495aa9434b800110f744218c0a to start an election
I20260709 13:14:03.369447 10466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73845cf648b3497c9309033a7379b5be"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
 from {username='slave'} at 127.0.0.1:51496
I20260709 13:14:03.369969 10466 raft_consensus.cc:493] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:03.370328 10466 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.374959 10541 raft_consensus.cc:1275] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.377426 10466 raft_consensus.cc:515] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.378567 10676 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:03.380076 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:03.380602 10390 raft_consensus.cc:3055] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:03.380910 10390 raft_consensus.cc:740] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:14:03.381587 10390 consensus_queue.cc:260] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.382539 10390 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.389786 10390 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:03.390926 10433 leader_election.cc:304] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c412495aa9434b800110f744218c0a, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:03.391685 10676 raft_consensus.cc:2804] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:03.391891 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "c6c412495aa9434b800110f744218c0a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:03.392196 10676 raft_consensus.cc:697] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Leader. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:03.392519 10540 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:03.393004 10676 consensus_queue.cc:237] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:03.397636 10540 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c412495aa9434b800110f744218c0a in term 3.
I20260709 13:14:03.398653 10466 leader_election.cc:290] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [CANDIDATE]: Term 3 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:14:03.401922 10286 catalog_manager.cc:5697] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a reported cstate change: term changed from 2 to 3, leader changed from cd0332521e0b4db09972c056204d24e3 (127.10.4.65) to c6c412495aa9434b800110f744218c0a (127.10.4.66). New cstate: current_term: 3 leader_uuid: "c6c412495aa9434b800110f744218c0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:03.490972 10390 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 5 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:03.492120 10676 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:03.498657 10540 raft_consensus.cc:1275] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 5 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:03.500715 10697 consensus_queue.cc:1048] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:03.597388 10390 raft_consensus.cc:1275] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.598618 10697 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:03.605305 10540 raft_consensus.cc:1275] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.606652 10705 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:03.618932 10390 raft_consensus.cc:1275] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.620064 10676 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:03.628135 10540 raft_consensus.cc:1275] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.630899 10676 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:03.839557 10390 raft_consensus.cc:1275] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.840781 10676 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:03.849282 10540 raft_consensus.cc:1275] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Refusing update from remote peer c6c412495aa9434b800110f744218c0a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.850664 10710 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:05.958382 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:05.965734 10466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
mode: GRACEFUL
new_leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:50726
I20260709 13:14:05.966307 10466 raft_consensus.cc:606] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Received request to transfer leadership to TS 98ef1ab89307452d8cd1dfa48d30564c
I20260709 13:14:05.968444 10466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
mode: GRACEFUL
new_leader_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:50726
I20260709 13:14:05.968811 10466 raft_consensus.cc:606] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Received request to transfer leadership to TS cd0332521e0b4db09972c056204d24e3
I20260709 13:14:05.970602 10466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73845cf648b3497c9309033a7379b5be"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
mode: GRACEFUL
new_leader_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:50726
I20260709 13:14:05.970945 10466 raft_consensus.cc:606] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Received request to transfer leadership to TS cd0332521e0b4db09972c056204d24e3
I20260709 13:14:05.972826 10466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
mode: GRACEFUL
new_leader_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:50726
I20260709 13:14:05.973160 10466 raft_consensus.cc:606] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Received request to transfer leadership to TS cd0332521e0b4db09972c056204d24e3
I20260709 13:14:05.974973 10466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39"
dest_uuid: "c6c412495aa9434b800110f744218c0a"
mode: GRACEFUL
new_leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:50726
I20260709 13:14:05.975315 10466 raft_consensus.cc:606] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Received request to transfer leadership to TS 98ef1ab89307452d8cd1dfa48d30564c
I20260709 13:14:05.976178 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:05.976542 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:06.148898 10697 raft_consensus.cc:993] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a: : Instructing follower cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:14:06.149372 10697 raft_consensus.cc:1081] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Signalling peer cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:14:06.150871 10390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:37250
I20260709 13:14:06.151283 10390 raft_consensus.cc:493] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:06.151644 10390 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.152338 10718 raft_consensus.cc:993] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a: : Instructing follower 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:06.152746 10697 raft_consensus.cc:1081] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Signalling peer 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:06.156960 10390 raft_consensus.cc:515] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:06.159703 10541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:06.159762 10540 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:43290
I20260709 13:14:06.159767 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eba7c08f8294327b314adc5ac590b05" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:14:06.160303 10541 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.160501 10466 raft_consensus.cc:3055] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:06.160466 10540 raft_consensus.cc:493] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:06.160951 10466 raft_consensus.cc:740] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:06.161042 10540 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.161798 10466 consensus_queue.cc:260] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:06.162868 10466 raft_consensus.cc:3060] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.165665 10541 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 4.
I20260709 13:14:06.166265 10390 leader_election.cc:290] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 4 election: Requested vote from peers 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:14:06.166932 10357 leader_election.cc:304] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:06.168196 10540 raft_consensus.cc:515] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.169052 10466 raft_consensus.cc:2468] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 4.
I20260709 13:14:06.170253 10540 leader_election.cc:290] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 4 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:14:06.171305 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:14:06.171859 10466 raft_consensus.cc:3055] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:06.171761 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030176ba7ebe4dc381240a987d5fddb4" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:06.172345 10390 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.172907 10719 raft_consensus.cc:2804] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:06.177981 10719 raft_consensus.cc:697] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 4 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:14:06.178128 10390 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 4.
I20260709 13:14:06.178673 10466 raft_consensus.cc:740] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:06.179049 10719 consensus_queue.cc:237] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } }
I20260709 13:14:06.179565 10507 leader_election.cc:304] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:06.179580 10466 consensus_queue.cc:260] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.180423 10715 raft_consensus.cc:2804] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:06.180831 10466 raft_consensus.cc:3060] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.181677 10715 raft_consensus.cc:697] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:14:06.182585 10715 consensus_queue.cc:237] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.190474 10466 raft_consensus.cc:2468] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 4.
I20260709 13:14:06.196144 10286 catalog_manager.cc:5697] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 3 to 4, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 4 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:06.196182 10289 catalog_manager.cc:5697] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 3 to 4, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 4 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:06.258829 10697 raft_consensus.cc:993] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: : Instructing follower 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:06.259215 10718 raft_consensus.cc:1081] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Signalling peer 98ef1ab89307452d8cd1dfa48d30564c to start an election
I20260709 13:14:06.260584 10540 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39"
dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
 from {username='slave'} at 127.0.0.1:43290
I20260709 13:14:06.261026 10540 raft_consensus.cc:493] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:06.261289 10540 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.265357 10540 raft_consensus.cc:515] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.266822 10540 leader_election.cc:290] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 4 election: Requested vote from peers cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327), c6c412495aa9434b800110f744218c0a (127.10.4.66:42071)
I20260709 13:14:06.267817 10390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cd0332521e0b4db09972c056204d24e3"
I20260709 13:14:06.267896 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b8957e7eef43a79f798a63434e9b39" candidate_uuid: "98ef1ab89307452d8cd1dfa48d30564c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:14:06.268373 10390 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.268440 10466 raft_consensus.cc:3055] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:06.268738 10466 raft_consensus.cc:740] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:06.269362 10466 consensus_queue.cc:260] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.270217 10466 raft_consensus.cc:3060] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.272570 10390 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 4.
I20260709 13:14:06.273522 10507 leader_election.cc:304] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:06.274344 10715 raft_consensus.cc:2804] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:06.274673 10466 raft_consensus.cc:2468] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ef1ab89307452d8cd1dfa48d30564c in term 4.
I20260709 13:14:06.274765 10715 raft_consensus.cc:697] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Becoming Leader. State: Replica: 98ef1ab89307452d8cd1dfa48d30564c, State: Running, Role: LEADER
I20260709 13:14:06.275755 10715 consensus_queue.cc:237] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.282369 10286 catalog_manager.cc:5697] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c reported cstate change: term changed from 3 to 4, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67). New cstate: current_term: 4 leader_uuid: "98ef1ab89307452d8cd1dfa48d30564c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:06.307812 10697 raft_consensus.cc:993] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a: : Instructing follower cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:14:06.308244 10718 raft_consensus.cc:1081] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Signalling peer cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:14:06.309734 10390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:37250
I20260709 13:14:06.310220 10390 raft_consensus.cc:493] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:06.310472 10390 raft_consensus.cc:3060] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:06.314518 10390 raft_consensus.cc:515] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.315891 10390 leader_election.cc:290] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Requested vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:14:06.317277 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:06.317436 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43b36a3a288649daa015b4bd91c84674" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:14:06.317867 10466 raft_consensus.cc:3055] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:06.317828 10540 raft_consensus.cc:3060] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:06.318238 10466 raft_consensus.cc:740] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:06.318900 10466 consensus_queue.cc:260] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.319970 10466 raft_consensus.cc:3060] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:06.322592 10540 raft_consensus.cc:2468] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:14:06.323609 10357 leader_election.cc:304] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:06.324266 10719 raft_consensus.cc:2804] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:06.324523 10466 raft_consensus.cc:2468] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 2.
I20260709 13:14:06.324631 10719 raft_consensus.cc:697] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:14:06.325542 10719 consensus_queue.cc:237] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.333168 10286 catalog_manager.cc:5697] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 1 to 2, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:06.359459 10697 raft_consensus.cc:993] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a: : Instructing follower cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:14:06.359859 10718 raft_consensus.cc:1081] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Signalling peer cd0332521e0b4db09972c056204d24e3 to start an election
I20260709 13:14:06.361300 10390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73845cf648b3497c9309033a7379b5be"
dest_uuid: "cd0332521e0b4db09972c056204d24e3"
 from {username='slave'} at 127.0.0.1:37250
I20260709 13:14:06.361713 10390 raft_consensus.cc:493] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:06.361953 10390 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.365757 10390 raft_consensus.cc:515] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.367211 10390 leader_election.cc:290] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 4 election: Requested vote from peers c6c412495aa9434b800110f744218c0a (127.10.4.66:42071), 98ef1ab89307452d8cd1dfa48d30564c (127.10.4.67:38229)
I20260709 13:14:06.368105 10466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c6c412495aa9434b800110f744218c0a"
I20260709 13:14:06.368204 10540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73845cf648b3497c9309033a7379b5be" candidate_uuid: "cd0332521e0b4db09972c056204d24e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "98ef1ab89307452d8cd1dfa48d30564c"
I20260709 13:14:06.368736 10466 raft_consensus.cc:3055] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:06.368790 10540 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.368994 10466 raft_consensus.cc:740] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c6c412495aa9434b800110f744218c0a, State: Running, Role: LEADER
I20260709 13:14:06.369728 10466 consensus_queue.cc:260] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.370563 10466 raft_consensus.cc:3060] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.373626 10540 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 4.
I20260709 13:14:06.374773 10357 leader_election.cc:304] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98ef1ab89307452d8cd1dfa48d30564c, cd0332521e0b4db09972c056204d24e3; no voters: 
I20260709 13:14:06.375326 10466 raft_consensus.cc:2468] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0332521e0b4db09972c056204d24e3 in term 4.
I20260709 13:14:06.375496 10719 raft_consensus.cc:2804] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:06.375998 10719 raft_consensus.cc:697] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 4 LEADER]: Becoming Leader. State: Replica: cd0332521e0b4db09972c056204d24e3, State: Running, Role: LEADER
I20260709 13:14:06.376857 10719 consensus_queue.cc:237] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } }
I20260709 13:14:06.384028 10286 catalog_manager.cc:5697] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 reported cstate change: term changed from 3 to 4, leader changed from c6c412495aa9434b800110f744218c0a (127.10.4.66) to cd0332521e0b4db09972c056204d24e3 (127.10.4.65). New cstate: current_term: 4 leader_uuid: "cd0332521e0b4db09972c056204d24e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:06.644068 10390 raft_consensus.cc:1275] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:06.645215 10715 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:06.656476 10466 raft_consensus.cc:1275] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:06.657822 10715 consensus_queue.cc:1048] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:06.731127 10466 raft_consensus.cc:1275] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:06.732321 10719 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:06.741901 10540 raft_consensus.cc:1275] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:06.743131 10719 consensus_queue.cc:1048] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:06.755154 10390 raft_consensus.cc:1275] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:06.756320 10715 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0332521e0b4db09972c056204d24e3" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:06.764428 10466 raft_consensus.cc:1275] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Refusing update from remote peer 98ef1ab89307452d8cd1dfa48d30564c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:06.765862 10715 consensus_queue.cc:1048] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:06.833017 10540 raft_consensus.cc:1275] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:06.834311 10719 consensus_queue.cc:1048] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:06.844281 10466 raft_consensus.cc:1275] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:06.845444 10744 consensus_queue.cc:1048] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:06.977370 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:06.982443 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
I20260709 13:14:06.990623 10540 raft_consensus.cc:1275] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:06.991940 10744 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ef1ab89307452d8cd1dfa48d30564c" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:06.999550 10466 raft_consensus.cc:1275] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Refusing update from remote peer cd0332521e0b4db09972c056204d24e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:07.001439 10744 consensus_queue.cc:1048] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c412495aa9434b800110f744218c0a" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:07.032797 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:07.033497 10257 tablet_replica.cc:333] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.034162 10257 raft_consensus.cc:2243] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.034663 10257 raft_consensus.cc:2272] T 0a54d74c40c043b8a6daab30bc33d4fd P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.036804 10257 tablet_replica.cc:333] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.037297 10257 raft_consensus.cc:2243] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.037665 10257 raft_consensus.cc:2272] T 1340349398384547b61df98ecaf81f90 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.039206 10257 tablet_replica.cc:333] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.039691 10257 raft_consensus.cc:2243] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.040258 10257 raft_consensus.cc:2272] T c3836d83f3cc483b8efb7ebd8e67dfd1 P cd0332521e0b4db09972c056204d24e3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.041875 10257 tablet_replica.cc:333] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.042312 10257 raft_consensus.cc:2243] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.042954 10257 raft_consensus.cc:2272] T 43b36a3a288649daa015b4bd91c84674 P cd0332521e0b4db09972c056204d24e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.044890 10257 tablet_replica.cc:333] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.045377 10257 raft_consensus.cc:2243] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.046029 10257 raft_consensus.cc:2272] T 73845cf648b3497c9309033a7379b5be P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.047950 10257 tablet_replica.cc:333] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.048372 10257 raft_consensus.cc:2243] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.048719 10257 raft_consensus.cc:2272] T 030176ba7ebe4dc381240a987d5fddb4 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.050627 10257 tablet_replica.cc:333] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.051051 10257 raft_consensus.cc:2243] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.051472 10257 raft_consensus.cc:2272] T e5b8957e7eef43a79f798a63434e9b39 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.053463 10257 tablet_replica.cc:333] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.053925 10257 raft_consensus.cc:2243] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.054270 10257 raft_consensus.cc:2272] T 597045aaee954382839393fe2ac62d46 P cd0332521e0b4db09972c056204d24e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.055855 10257 tablet_replica.cc:333] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3: stopping tablet replica
I20260709 13:14:07.056263 10257 raft_consensus.cc:2243] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.056893 10257 raft_consensus.cc:2272] T 7eba7c08f8294327b314adc5ac590b05 P cd0332521e0b4db09972c056204d24e3 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.157605 10433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.65:34327: connect: Connection refused (error 111)
I20260709 13:14:07.159158 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
W20260709 13:14:07.161389 10433 consensus_peers.cc:597] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a -> Peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Couldn't send request to peer cd0332521e0b4db09972c056204d24e3. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:34327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.176535 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
I20260709 13:14:07.195272 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:07.195892 10257 tablet_replica.cc:333] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.196635 10257 raft_consensus.cc:2243] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.197181 10257 raft_consensus.cc:2272] T 1340349398384547b61df98ecaf81f90 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.199100 10257 tablet_replica.cc:333] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.199644 10257 raft_consensus.cc:2243] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.200484 10257 raft_consensus.cc:2272] T 0a54d74c40c043b8a6daab30bc33d4fd P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:07.202158 10507 consensus_peers.cc:597] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c -> Peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Couldn't send request to peer cd0332521e0b4db09972c056204d24e3. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:34327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.202412 10257 tablet_replica.cc:333] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.202894 10257 raft_consensus.cc:2243] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.203255 10257 raft_consensus.cc:2272] T 73845cf648b3497c9309033a7379b5be P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.205097 10257 tablet_replica.cc:333] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.205581 10257 raft_consensus.cc:2243] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.205955 10257 raft_consensus.cc:2272] T e5b8957e7eef43a79f798a63434e9b39 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.207938 10257 tablet_replica.cc:333] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.208386 10257 raft_consensus.cc:2243] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.208740 10257 raft_consensus.cc:2272] T 030176ba7ebe4dc381240a987d5fddb4 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.210522 10257 tablet_replica.cc:333] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.210983 10257 raft_consensus.cc:2243] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.211364 10257 raft_consensus.cc:2272] T 7eba7c08f8294327b314adc5ac590b05 P c6c412495aa9434b800110f744218c0a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.213191 10257 tablet_replica.cc:333] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.213641 10257 raft_consensus.cc:2243] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.214325 10257 raft_consensus.cc:2272] T 597045aaee954382839393fe2ac62d46 P c6c412495aa9434b800110f744218c0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.215901 10257 tablet_replica.cc:333] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.216322 10257 raft_consensus.cc:2243] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.216670 10257 raft_consensus.cc:2272] T 43b36a3a288649daa015b4bd91c84674 P c6c412495aa9434b800110f744218c0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.218667 10257 tablet_replica.cc:333] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a: stopping tablet replica
I20260709 13:14:07.219064 10257 raft_consensus.cc:2243] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.219882 10257 raft_consensus.cc:2272] T c3836d83f3cc483b8efb7ebd8e67dfd1 P c6c412495aa9434b800110f744218c0a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.245209 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
I20260709 13:14:07.261641 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
W20260709 13:14:07.265398 10507 consensus_peers.cc:597] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c -> Peer cd0332521e0b4db09972c056204d24e3 (127.10.4.65:34327): Couldn't send request to peer cd0332521e0b4db09972c056204d24e3. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:34327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:07.281275 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:07.281873 10257 tablet_replica.cc:333] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.282359 10257 raft_consensus.cc:2243] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.283038 10257 raft_consensus.cc:2272] T 1340349398384547b61df98ecaf81f90 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.284821 10257 tablet_replica.cc:333] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.285274 10257 raft_consensus.cc:2243] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.285629 10257 raft_consensus.cc:2272] T c3836d83f3cc483b8efb7ebd8e67dfd1 P 98ef1ab89307452d8cd1dfa48d30564c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.287137 10257 tablet_replica.cc:333] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.287618 10257 raft_consensus.cc:2243] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.287973 10257 raft_consensus.cc:2272] T 0a54d74c40c043b8a6daab30bc33d4fd P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.289408 10257 tablet_replica.cc:333] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.289814 10257 raft_consensus.cc:2243] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.290156 10257 raft_consensus.cc:2272] T 7eba7c08f8294327b314adc5ac590b05 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.292081 10257 tablet_replica.cc:333] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.292496 10257 raft_consensus.cc:2243] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.292819 10257 raft_consensus.cc:2272] T 597045aaee954382839393fe2ac62d46 P 98ef1ab89307452d8cd1dfa48d30564c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.294265 10257 tablet_replica.cc:333] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.294642 10257 raft_consensus.cc:2243] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.295274 10257 raft_consensus.cc:2272] T e5b8957e7eef43a79f798a63434e9b39 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.297432 10257 tablet_replica.cc:333] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.297852 10257 raft_consensus.cc:2243] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.298208 10257 raft_consensus.cc:2272] T 73845cf648b3497c9309033a7379b5be P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.300019 10257 tablet_replica.cc:333] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.300431 10257 raft_consensus.cc:2243] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.301064 10257 raft_consensus.cc:2272] T 030176ba7ebe4dc381240a987d5fddb4 P 98ef1ab89307452d8cd1dfa48d30564c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.302788 10257 tablet_replica.cc:333] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c: stopping tablet replica
I20260709 13:14:07.303157 10257 raft_consensus.cc:2243] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:07.303515 10257 raft_consensus.cc:2272] T 43b36a3a288649daa015b4bd91c84674 P 98ef1ab89307452d8cd1dfa48d30564c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.328087 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
I20260709 13:14:07.342814 10257 master.cc:562] Master@127.10.4.126:40673 shutting down...
I20260709 13:14:07.546058 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:07.546593 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:07.546897 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P f7e8bd805c6e404da4e56529d24db1cf: stopping tablet replica
I20260709 13:14:07.565112 10257 master.cc:584] Master@127.10.4.126:40673 shutdown complete.
98ef1ab89307452d8cd1dfa48d30564c  1
c6c412495aa9434b800110f744218c0a  4
cd0332521e0b4db09972c056204d24e3  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12498 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:14:07.606288 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:34809
I20260709 13:14:07.607363 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:07.612591 10753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:07.612954 10752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:07.613866 10257 server_base.cc:1061] running on GCE node
W20260709 13:14:07.614288 10755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:07.615020 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:07.615197 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:07.615332 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602847615315 us; error 0 us; skew 500 ppm
I20260709 13:14:07.615854 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:07.618189 10257 webserver.cc:533] Webserver started at http://127.10.4.126:40097/ using document root <none> and password file <none>
I20260709 13:14:07.618656 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:07.618816 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:07.619064 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:07.620214 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "3e8a5b1ca7de4591a8761fc397773025"
format_stamp: "Formatted at 2026-07-09 13:14:07 on dist-test-slave-gkm9"
I20260709 13:14:07.624284 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:14:07.627260 10760 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:07.627974 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:07.628244 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "3e8a5b1ca7de4591a8761fc397773025"
format_stamp: "Formatted at 2026-07-09 13:14:07 on dist-test-slave-gkm9"
I20260709 13:14:07.628492 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:07.647583 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:07.648814 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:07.686668 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:34809
I20260709 13:14:07.686789 10811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:34809 every 8 connection(s)
I20260709 13:14:07.690279 10812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:07.699491 10812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025: Bootstrap starting.
I20260709 13:14:07.704114 10812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:07.708250 10812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025: No bootstrap required, opened a new log
I20260709 13:14:07.710274 10812 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8a5b1ca7de4591a8761fc397773025" member_type: VOTER }
I20260709 13:14:07.710629 10812 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:07.710843 10812 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8a5b1ca7de4591a8761fc397773025, State: Initialized, Role: FOLLOWER
I20260709 13:14:07.711393 10812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8a5b1ca7de4591a8761fc397773025" member_type: VOTER }
I20260709 13:14:07.711844 10812 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:07.712054 10812 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:07.712313 10812 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:07.716161 10812 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8a5b1ca7de4591a8761fc397773025" member_type: VOTER }
I20260709 13:14:07.716626 10812 leader_election.cc:304] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e8a5b1ca7de4591a8761fc397773025; no voters: 
I20260709 13:14:07.717661 10812 leader_election.cc:290] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:07.718017 10815 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:07.719161 10815 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 1 LEADER]: Becoming Leader. State: Replica: 3e8a5b1ca7de4591a8761fc397773025, State: Running, Role: LEADER
I20260709 13:14:07.719817 10815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8a5b1ca7de4591a8761fc397773025" member_type: VOTER }
I20260709 13:14:07.720369 10812 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:07.723881 10816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e8a5b1ca7de4591a8761fc397773025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8a5b1ca7de4591a8761fc397773025" member_type: VOTER } }
I20260709 13:14:07.724177 10817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e8a5b1ca7de4591a8761fc397773025. Latest consensus state: current_term: 1 leader_uuid: "3e8a5b1ca7de4591a8761fc397773025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8a5b1ca7de4591a8761fc397773025" member_type: VOTER } }
I20260709 13:14:07.724599 10816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:07.724750 10817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:07.729252 10822 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:07.734833 10822 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:07.735754 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:07.743347 10822 catalog_manager.cc:1383] Generated new cluster ID: fdee46a412b343ef81eead546bc8e7aa
I20260709 13:14:07.743631 10822 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:07.764279 10822 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:07.765467 10822 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:07.777587 10822 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025: Generated new TSK 0
I20260709 13:14:07.778155 10822 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:07.802448 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:07.808147 10833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:07.809165 10834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:07.810719 10836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:07.811038 10257 server_base.cc:1061] running on GCE node
I20260709 13:14:07.811786 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:07.811967 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:07.812112 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602847812096 us; error 0 us; skew 500 ppm
I20260709 13:14:07.812577 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:07.814602 10257 webserver.cc:533] Webserver started at http://127.10.4.65:46773/ using document root <none> and password file <none>
I20260709 13:14:07.815048 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:07.815217 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:07.815495 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:07.816460 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "d86d8ebbcaff4e76af8e3acb67207878"
format_stamp: "Formatted at 2026-07-09 13:14:07 on dist-test-slave-gkm9"
I20260709 13:14:07.820501 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:14:07.823520 10841 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:07.824188 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:07.824431 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "d86d8ebbcaff4e76af8e3acb67207878"
format_stamp: "Formatted at 2026-07-09 13:14:07 on dist-test-slave-gkm9"
I20260709 13:14:07.824697 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:07.838214 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:07.839903 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:07.846570 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:07.849861 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:07.850143 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:07.850448 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:07.850677 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:07.896108 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:44757
I20260709 13:14:07.896211 10903 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:44757 every 8 connection(s)
I20260709 13:14:07.900781 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:07.907644 10908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:07.908217 10909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:07.911831 10257 server_base.cc:1061] running on GCE node
W20260709 13:14:07.919189 10911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:07.920380 10904 heartbeater.cc:344] Connected to a master server at 127.10.4.126:34809
I20260709 13:14:07.920656 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:14:07.920809 10904 heartbeater.cc:461] Registering TS with master...
W20260709 13:14:07.920938 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:07.921140 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602847921121 us; error 0 us; skew 500 ppm
I20260709 13:14:07.921736 10904 heartbeater.cc:507] Master 127.10.4.126:34809 requested a full tablet report, sending...
I20260709 13:14:07.921859 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:07.924254 10777 ts_manager.cc:194] Registered new tserver with Master: d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757)
I20260709 13:14:07.926069 10777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38676
I20260709 13:14:07.928195 10257 webserver.cc:533] Webserver started at http://127.10.4.66:39617/ using document root <none> and password file <none>
I20260709 13:14:07.928785 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:07.929021 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:07.929325 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:07.930745 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
format_stamp: "Formatted at 2026-07-09 13:14:07 on dist-test-slave-gkm9"
I20260709 13:14:07.934828 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:14:07.938123 10916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:07.938802 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:14:07.939051 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
format_stamp: "Formatted at 2026-07-09 13:14:07 on dist-test-slave-gkm9"
I20260709 13:14:07.939316 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:07.952600 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:07.953742 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:07.955137 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:07.957191 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:07.957374 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:07.957577 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:07.957727 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:07.994829 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:34223
I20260709 13:14:07.994916 10978 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:34223 every 8 connection(s)
I20260709 13:14:07.999099 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:08.006309 10982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:08.007128 10983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:08.009163 10985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:08.009733 10257 server_base.cc:1061] running on GCE node
I20260709 13:14:08.010715 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:08.010912 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:08.011096 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602848011079 us; error 0 us; skew 500 ppm
I20260709 13:14:08.011687 10979 heartbeater.cc:344] Connected to a master server at 127.10.4.126:34809
I20260709 13:14:08.011801 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:08.012063 10979 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:08.012945 10979 heartbeater.cc:507] Master 127.10.4.126:34809 requested a full tablet report, sending...
I20260709 13:14:08.014392 10257 webserver.cc:533] Webserver started at http://127.10.4.67:35513/ using document root <none> and password file <none>
I20260709 13:14:08.014919 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:08.015079 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:08.015045 10777 ts_manager.cc:194] Registered new tserver with Master: fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.015473 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:08.016466 10777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38692
I20260709 13:14:08.016522 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "e17dd92ad52846bd93d4c3237acd28e0"
format_stamp: "Formatted at 2026-07-09 13:14:08 on dist-test-slave-gkm9"
I20260709 13:14:08.020795 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:14:08.023820 10990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:08.024489 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:14:08.024748 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "e17dd92ad52846bd93d4c3237acd28e0"
format_stamp: "Formatted at 2026-07-09 13:14:08 on dist-test-slave-gkm9"
I20260709 13:14:08.024992 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:08.072636 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:08.073776 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:08.075183 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:08.077291 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:08.077476 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:08.077687 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:08.077833 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:08.115357 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:44365
I20260709 13:14:08.115486 11052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:44365 every 8 connection(s)
I20260709 13:14:08.128741 11053 heartbeater.cc:344] Connected to a master server at 127.10.4.126:34809
I20260709 13:14:08.129084 11053 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:08.129755 11053 heartbeater.cc:507] Master 127.10.4.126:34809 requested a full tablet report, sending...
I20260709 13:14:08.131579 10777 ts_manager.cc:194] Registered new tserver with Master: e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.131709 10257 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013221925s
I20260709 13:14:08.132848 10777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38702
I20260709 13:14:08.154789 10777 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38718:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:08.157653 10777 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:08.237483 10944 tablet_service.cc:1511] Processing CreateTablet for tablet 42e64608e7c94b7caa8664d7aed15b50 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:08.238865 10944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e64608e7c94b7caa8664d7aed15b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.241125 10943 tablet_service.cc:1511] Processing CreateTablet for tablet 0658ee47d6e4413e9f207532ea4999cc (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:08.242436 10943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0658ee47d6e4413e9f207532ea4999cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.251075 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 42e64608e7c94b7caa8664d7aed15b50 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:08.251349 10868 tablet_service.cc:1511] Processing CreateTablet for tablet 0658ee47d6e4413e9f207532ea4999cc (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:08.252492 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e64608e7c94b7caa8664d7aed15b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.253194 10868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0658ee47d6e4413e9f207532ea4999cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.253189 10942 tablet_service.cc:1511] Processing CreateTablet for tablet aada48f8534745c19e54a404fc4a3037 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:08.254709 10942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aada48f8534745c19e54a404fc4a3037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.262214 10938 tablet_service.cc:1511] Processing CreateTablet for tablet 93e7d762c4d04bc0a93f865c0e4fffa9 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:08.262723 10941 tablet_service.cc:1511] Processing CreateTablet for tablet 7946fded4dfd4022b386d9485b8ca5ce (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:08.264557 11018 tablet_service.cc:1511] Processing CreateTablet for tablet 42e64608e7c94b7caa8664d7aed15b50 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:14:08.265832 11018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e64608e7c94b7caa8664d7aed15b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.267684 10935 tablet_service.cc:1511] Processing CreateTablet for tablet 3658791733f84b198f0385c5dac7cca2 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:08.268675 11017 tablet_service.cc:1511] Processing CreateTablet for tablet 0658ee47d6e4413e9f207532ea4999cc (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:14:08.270064 11017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0658ee47d6e4413e9f207532ea4999cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.272609 11016 tablet_service.cc:1511] Processing CreateTablet for tablet aada48f8534745c19e54a404fc4a3037 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:08.274899 11015 tablet_service.cc:1511] Processing CreateTablet for tablet 7946fded4dfd4022b386d9485b8ca5ce (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:08.277196 11014 tablet_service.cc:1511] Processing CreateTablet for tablet 34941245c64f41ffb49311a76630551a (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:08.279670 10940 tablet_service.cc:1511] Processing CreateTablet for tablet 34941245c64f41ffb49311a76630551a (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:08.285472 11012 tablet_service.cc:1511] Processing CreateTablet for tablet 93e7d762c4d04bc0a93f865c0e4fffa9 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:08.287964 10937 tablet_service.cc:1511] Processing CreateTablet for tablet abd752c0c8fd42c0be6146dc0a7d1dbf (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:08.290337 10939 tablet_service.cc:1511] Processing CreateTablet for tablet 8634103b437b49a8b0d8724e44da474a (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:08.292623 11011 tablet_service.cc:1511] Processing CreateTablet for tablet abd752c0c8fd42c0be6146dc0a7d1dbf (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:08.295630 11010 tablet_service.cc:1511] Processing CreateTablet for tablet a1bb61541b6b4b6f9d6be0312ba07c81 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:08.298710 11009 tablet_service.cc:1511] Processing CreateTablet for tablet 3658791733f84b198f0385c5dac7cca2 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:08.265471 10936 tablet_service.cc:1511] Processing CreateTablet for tablet a1bb61541b6b4b6f9d6be0312ba07c81 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:08.282253 11013 tablet_service.cc:1511] Processing CreateTablet for tablet 8634103b437b49a8b0d8724e44da474a (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:08.301386 10938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93e7d762c4d04bc0a93f865c0e4fffa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.301889 11068 tablet_bootstrap.cc:492] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.301980 10941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7946fded4dfd4022b386d9485b8ca5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.302964 11069 tablet_bootstrap.cc:492] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.304100 11016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aada48f8534745c19e54a404fc4a3037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.304188 10935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3658791733f84b198f0385c5dac7cca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.305949 11013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8634103b437b49a8b0d8724e44da474a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.306646 11011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd752c0c8fd42c0be6146dc0a7d1dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.307231 11014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34941245c64f41ffb49311a76630551a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.307776 10939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8634103b437b49a8b0d8724e44da474a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.308493 11009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3658791733f84b198f0385c5dac7cca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.311736 11012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93e7d762c4d04bc0a93f865c0e4fffa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.311739 10936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1bb61541b6b4b6f9d6be0312ba07c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.314271 11010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1bb61541b6b4b6f9d6be0312ba07c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.315692 10937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd752c0c8fd42c0be6146dc0a7d1dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.319754 11015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7946fded4dfd4022b386d9485b8ca5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.321058 11068 tablet_bootstrap.cc:654] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.337555 10867 tablet_service.cc:1511] Processing CreateTablet for tablet aada48f8534745c19e54a404fc4a3037 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:14:08.338953 10867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aada48f8534745c19e54a404fc4a3037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.340663 10940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34941245c64f41ffb49311a76630551a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.361194 10869 tablet_service.cc:1511] Processing CreateTablet for tablet 7946fded4dfd4022b386d9485b8ca5ce (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:14:08.362486 10869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7946fded4dfd4022b386d9485b8ca5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.367547 10866 tablet_service.cc:1511] Processing CreateTablet for tablet 34941245c64f41ffb49311a76630551a (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:14:08.369020 10866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34941245c64f41ffb49311a76630551a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.378093 10865 tablet_service.cc:1511] Processing CreateTablet for tablet 8634103b437b49a8b0d8724e44da474a (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:14:08.378832 10863 tablet_service.cc:1511] Processing CreateTablet for tablet abd752c0c8fd42c0be6146dc0a7d1dbf (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:14:08.379380 10865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8634103b437b49a8b0d8724e44da474a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.379874 11069 tablet_bootstrap.cc:654] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.380172 10863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd752c0c8fd42c0be6146dc0a7d1dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.388281 10862 tablet_service.cc:1511] Processing CreateTablet for tablet a1bb61541b6b4b6f9d6be0312ba07c81 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:14:08.389710 10862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1bb61541b6b4b6f9d6be0312ba07c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.391546 10864 tablet_service.cc:1511] Processing CreateTablet for tablet 93e7d762c4d04bc0a93f865c0e4fffa9 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:14:08.392894 10864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93e7d762c4d04bc0a93f865c0e4fffa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.427359 10861 tablet_service.cc:1511] Processing CreateTablet for tablet 3658791733f84b198f0385c5dac7cca2 (DEFAULT_TABLE table=test-workload [id=e1bc6dd0bbb243699f145ef52c18be78]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:14:08.428642 10861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3658791733f84b198f0385c5dac7cca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:08.448058 11070 tablet_bootstrap.cc:492] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:08.465230 11069 tablet_bootstrap.cc:492] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.465737 11069 ts_tablet_manager.cc:1403] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.163s	user 0.019s	sys 0.031s
I20260709 13:14:08.468556 11069 raft_consensus.cc:359] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.469225 11069 raft_consensus.cc:385] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.469507 11069 raft_consensus.cc:740] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.470185 11069 consensus_queue.cc:260] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.478543 11070 tablet_bootstrap.cc:654] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.478571 11068 tablet_bootstrap.cc:492] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.480580 10904 heartbeater.cc:499] Master 127.10.4.126:34809 was elected leader, sending a full tablet report...
I20260709 13:14:08.481024 11069 ts_tablet_manager.cc:1434] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.006s
I20260709 13:14:08.482033 11069 tablet_bootstrap.cc:492] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.484279 11070 tablet_bootstrap.cc:492] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.484822 11070 ts_tablet_manager.cc:1403] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.000s
I20260709 13:14:08.486244 11068 ts_tablet_manager.cc:1403] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.178s	user 0.012s	sys 0.006s
I20260709 13:14:08.488014 11070 raft_consensus.cc:359] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.488708 11069 tablet_bootstrap.cc:654] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.488790 11070 raft_consensus.cc:385] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.489264 11070 raft_consensus.cc:740] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.489176 11068 raft_consensus.cc:359] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.489940 11068 raft_consensus.cc:385] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.490272 11068 raft_consensus.cc:740] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.490384 11070 consensus_queue.cc:260] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.490998 11068 consensus_queue.cc:260] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.499419 10979 heartbeater.cc:499] Master 127.10.4.126:34809 was elected leader, sending a full tablet report...
I20260709 13:14:08.500463 11068 ts_tablet_manager.cc:1434] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:14:08.501549 11068 tablet_bootstrap.cc:492] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.504451 11069 tablet_bootstrap.cc:492] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.504928 11069 ts_tablet_manager.cc:1403] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.008s
I20260709 13:14:08.507756 11069 raft_consensus.cc:359] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.508457 11069 raft_consensus.cc:385] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.508736 11069 raft_consensus.cc:740] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.509150 11068 tablet_bootstrap.cc:654] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.509680 11069 consensus_queue.cc:260] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.512130 11069 ts_tablet_manager.cc:1434] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:08.513069 11069 tablet_bootstrap.cc:492] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.519259 11069 tablet_bootstrap.cc:654] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.521014 11068 tablet_bootstrap.cc:492] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.521499 11068 ts_tablet_manager.cc:1403] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.007s
I20260709 13:14:08.524878 11068 raft_consensus.cc:359] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.525681 11068 raft_consensus.cc:385] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.525991 11068 raft_consensus.cc:740] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.527161 11068 consensus_queue.cc:260] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.530031 11068 ts_tablet_manager.cc:1434] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.009s
I20260709 13:14:08.530834 11053 heartbeater.cc:499] Master 127.10.4.126:34809 was elected leader, sending a full tablet report...
I20260709 13:14:08.531292 11068 tablet_bootstrap.cc:492] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.532260 11070 ts_tablet_manager.cc:1434] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.047s	user 0.018s	sys 0.010s
I20260709 13:14:08.533959 11070 tablet_bootstrap.cc:492] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:08.537518 11069 tablet_bootstrap.cc:492] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.537984 11069 ts_tablet_manager.cc:1403] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.008s
I20260709 13:14:08.539251 11068 tablet_bootstrap.cc:654] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.540753 11069 raft_consensus.cc:359] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.541476 11069 raft_consensus.cc:385] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.530839 11076 raft_consensus.cc:493] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.541810 11069 raft_consensus.cc:740] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.542253 11076 raft_consensus.cc:515] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.542783 11069 consensus_queue.cc:260] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.544448 11070 tablet_bootstrap.cc:654] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.548125 11069 ts_tablet_manager.cc:1434] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:14:08.549208 11069 tablet_bootstrap.cc:492] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.550594 11068 tablet_bootstrap.cc:492] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.551049 11068 ts_tablet_manager.cc:1403] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 13:14:08.553869 11068 raft_consensus.cc:359] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.554603 11068 raft_consensus.cc:385] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.554891 11068 raft_consensus.cc:740] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.555357 11069 tablet_bootstrap.cc:654] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.555864 11068 consensus_queue.cc:260] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.556412 11070 tablet_bootstrap.cc:492] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.557022 11070 ts_tablet_manager.cc:1403] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.008s
I20260709 13:14:08.558454 11068 ts_tablet_manager.cc:1434] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:14:08.559556 11068 tablet_bootstrap.cc:492] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.560091 11070 raft_consensus.cc:359] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.560878 11070 raft_consensus.cc:385] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.561232 11070 raft_consensus.cc:740] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.562122 11070 consensus_queue.cc:260] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.564173 11070 ts_tablet_manager.cc:1434] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:08.565133 11070 tablet_bootstrap.cc:492] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:08.568197 11068 tablet_bootstrap.cc:654] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.571525 11070 tablet_bootstrap.cc:654] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.585506 11070 tablet_bootstrap.cc:492] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.586076 11070 ts_tablet_manager.cc:1403] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:14:08.588830 11070 raft_consensus.cc:359] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.589980 11070 raft_consensus.cc:385] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.590345 11070 raft_consensus.cc:740] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.591246 11070 consensus_queue.cc:260] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.594141 11070 ts_tablet_manager.cc:1434] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:14:08.597322 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.598223 11070 tablet_bootstrap.cc:492] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
W20260709 13:14:08.599237 10993 leader_election.cc:343] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.605058 11076 leader_election.cc:290] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.605260 11070 tablet_bootstrap.cc:654] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.610198 11073 raft_consensus.cc:493] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.610786 11073 raft_consensus.cc:515] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.615723 11073 leader_election.cc:290] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.618686 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
W20260709 13:14:08.620463 10992 leader_election.cc:343] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.620924 10992 leader_election.cc:304] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0; no voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:08.622013 11076 raft_consensus.cc:2749] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:08.631469 11069 tablet_bootstrap.cc:492] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.631975 11069 ts_tablet_manager.cc:1403] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.083s	user 0.029s	sys 0.037s
I20260709 13:14:08.634121 11068 tablet_bootstrap.cc:492] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.634644 11068 ts_tablet_manager.cc:1403] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.075s	user 0.043s	sys 0.021s
I20260709 13:14:08.635449 11069 raft_consensus.cc:359] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.636142 11069 raft_consensus.cc:385] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.636420 11069 raft_consensus.cc:740] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.637187 11069 consensus_queue.cc:260] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.637383 11068 raft_consensus.cc:359] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.638020 11068 raft_consensus.cc:385] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.638300 11068 raft_consensus.cc:740] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.639642 11070 tablet_bootstrap.cc:492] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.640130 11070 ts_tablet_manager.cc:1403] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.021s
I20260709 13:14:08.639930 11068 consensus_queue.cc:260] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.643334 11069 ts_tablet_manager.cc:1434] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:14:08.643181 11070 raft_consensus.cc:359] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.643894 11068 ts_tablet_manager.cc:1434] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:14:08.644055 11070 raft_consensus.cc:385] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.644382 11069 tablet_bootstrap.cc:492] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.644474 11070 raft_consensus.cc:740] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.644963 11068 tablet_bootstrap.cc:492] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.645569 11070 consensus_queue.cc:260] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.648595 11070 ts_tablet_manager.cc:1434] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:14:08.652297 11070 tablet_bootstrap.cc:492] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:08.657421 11077 raft_consensus.cc:493] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.657893 11077 raft_consensus.cc:515] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.658512 11073 raft_consensus.cc:493] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.659065 11073 raft_consensus.cc:515] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.660934 11070 tablet_bootstrap.cc:654] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.660871 11073 leader_election.cc:290] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.668144 11076 raft_consensus.cc:493] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.661618 11077 leader_election.cc:290] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.668759 11076 raft_consensus.cc:515] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.670789 11076 leader_election.cc:290] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.671542 11073 raft_consensus.cc:493] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.672459 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:08.678851 10954 raft_consensus.cc:2468] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:08.681059 11069 tablet_bootstrap.cc:654] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.680874 10992 leader_election.cc:304] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:08.681860 11076 raft_consensus.cc:2804] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:08.682220 11076 raft_consensus.cc:493] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.682549 11076 raft_consensus.cc:3060] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.672041 11073 raft_consensus.cc:515] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.684669 11073 leader_election.cc:290] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.685456 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:08.686208 10953 raft_consensus.cc:2468] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:08.687937 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3658791733f84b198f0385c5dac7cca2" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:08.688364 10843 leader_election.cc:304] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
W20260709 13:14:08.689460 10843 leader_election.cc:343] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.693674 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:08.694322 10954 raft_consensus.cc:2468] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:08.692772 11073 raft_consensus.cc:2804] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:08.698675 10843 leader_election.cc:304] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:08.700212 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.700906 10879 raft_consensus.cc:2468] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:08.698566 11073 raft_consensus.cc:493] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.700212 11076 raft_consensus.cc:515] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.707031 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:08.707140 11028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3658791733f84b198f0385c5dac7cca2" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:08.707996 11026 raft_consensus.cc:2393] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d86d8ebbcaff4e76af8e3acb67207878 in current term 1: Already voted for candidate e17dd92ad52846bd93d4c3237acd28e0 in this term.
I20260709 13:14:08.707056 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:08.709056 11027 raft_consensus.cc:2468] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:08.709530 11076 leader_election.cc:290] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.703560 11093 raft_consensus.cc:2804] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:14:08.709213 10843 leader_election.cc:343] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.710799 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:08.705627 11073 raft_consensus.cc:3060] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.711359 10954 raft_consensus.cc:3060] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.712302 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:08.712796 11093 raft_consensus.cc:493] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.713184 11093 raft_consensus.cc:3060] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.711110 10843 leader_election.cc:304] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878; no voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:08.723996 10954 raft_consensus.cc:2468] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 1.
I20260709 13:14:08.727819 11093 raft_consensus.cc:515] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.728883 10992 leader_election.cc:304] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:08.729938 10879 raft_consensus.cc:2393] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e17dd92ad52846bd93d4c3237acd28e0 in current term 1: Already voted for candidate d86d8ebbcaff4e76af8e3acb67207878 in this term.
W20260709 13:14:08.731002 10825 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:08.731002 11093 leader_election.cc:290] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.732093 11093 raft_consensus.cc:2749] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:08.732900 11076 raft_consensus.cc:2804] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:08.733368 11076 raft_consensus.cc:697] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:08.721016 11073 raft_consensus.cc:515] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.734114 11076 consensus_queue.cc:237] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.734609 11069 tablet_bootstrap.cc:492] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.735366 11069 ts_tablet_manager.cc:1403] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.091s	user 0.015s	sys 0.009s
I20260709 13:14:08.737324 11068 tablet_bootstrap.cc:654] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.740325 11069 raft_consensus.cc:359] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.741051 11069 raft_consensus.cc:385] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.741344 11069 raft_consensus.cc:740] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.742408 11069 consensus_queue.cc:260] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.743537 11070 tablet_bootstrap.cc:492] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.743985 11070 ts_tablet_manager.cc:1403] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.006s
I20260709 13:14:08.744522 11069 ts_tablet_manager.cc:1434] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:14:08.745189 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:08.745546 11069 tablet_bootstrap.cc:492] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.745734 11027 raft_consensus.cc:2468] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 0.
I20260709 13:14:08.746739 11070 raft_consensus.cc:359] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.747099 10918 leader_election.cc:304] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:08.747435 11070 raft_consensus.cc:385] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.747746 11070 raft_consensus.cc:740] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.747958 11077 raft_consensus.cc:2804] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:08.748328 11077 raft_consensus.cc:493] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.748872 11077 raft_consensus.cc:3060] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.748461 11070 consensus_queue.cc:260] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.751659 11070 ts_tablet_manager.cc:1434] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:14:08.752660 11070 tablet_bootstrap.cc:492] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:08.758692 11073 leader_election.cc:290] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.759271 11078 raft_consensus.cc:493] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.759851 11078 raft_consensus.cc:515] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.760618 11069 tablet_bootstrap.cc:654] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.761911 11078 leader_election.cc:290] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.763342 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:08.770140 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
W20260709 13:14:08.770313 10992 leader_election.cc:343] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.770910 10953 raft_consensus.cc:2393] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d86d8ebbcaff4e76af8e3acb67207878 in current term 1: Already voted for candidate e17dd92ad52846bd93d4c3237acd28e0 in this term.
I20260709 13:14:08.771348 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.772017 10879 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:08.773171 10993 leader_election.cc:304] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:08.774241 11078 raft_consensus.cc:2804] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:08.774631 11078 raft_consensus.cc:493] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.774314 11077 raft_consensus.cc:515] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.774989 11078 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.778004 11068 tablet_bootstrap.cc:492] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.778553 11068 ts_tablet_manager.cc:1403] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.134s	user 0.013s	sys 0.009s
I20260709 13:14:08.781518 11070 tablet_bootstrap.cc:654] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.781924 11068 raft_consensus.cc:359] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.782691 11068 raft_consensus.cc:385] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.783028 11068 raft_consensus.cc:740] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.782213 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:08.783847 11099 raft_consensus.cc:493] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.783921 11068 consensus_queue.cc:260] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.784829 11099 raft_consensus.cc:515] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.786700 11078 raft_consensus.cc:515] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.787248 10776 catalog_manager.cc:5697] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.789593 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:08.790488 10953 raft_consensus.cc:2393] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d86d8ebbcaff4e76af8e3acb67207878 in current term 1: Already voted for candidate fd3e554dff2d4ac1841f6d8c267b8578 in this term.
I20260709 13:14:08.785825 11069 tablet_bootstrap.cc:492] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.792330 11069 ts_tablet_manager.cc:1403] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.000s
I20260709 13:14:08.789626 11078 leader_election.cc:290] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.793919 11068 ts_tablet_manager.cc:1434] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.011s
I20260709 13:14:08.794812 11068 tablet_bootstrap.cc:492] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.789642 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:08.795302 10879 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.796396 10878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.797171 10878 raft_consensus.cc:2393] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd3e554dff2d4ac1841f6d8c267b8578 in current term 1: Already voted for candidate d86d8ebbcaff4e76af8e3acb67207878 in this term.
I20260709 13:14:08.798146 10877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:08.798914 10877 raft_consensus.cc:2393] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fd3e554dff2d4ac1841f6d8c267b8578 in current term 1: Already voted for candidate d86d8ebbcaff4e76af8e3acb67207878 in this term.
I20260709 13:14:08.799558 10843 leader_election.cc:304] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878; no voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:08.800663 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:08.799919 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.802269 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:08.804865 11027 raft_consensus.cc:3060] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.806682 11069 raft_consensus.cc:359] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.807351 11069 raft_consensus.cc:385] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.807657 11069 raft_consensus.cc:740] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.808351 11069 consensus_queue.cc:260] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.801630 11073 raft_consensus.cc:2749] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:08.812969 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
W20260709 13:14:08.805986 10992 leader_election.cc:343] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:08.814606 10918 leader_election.cc:343] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.815639 11099 leader_election.cc:290] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.817942 11068 tablet_bootstrap.cc:654] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.823038 11027 raft_consensus.cc:2468] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 1.
I20260709 13:14:08.823634 11077 leader_election.cc:290] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.824496 11101 raft_consensus.cc:493] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.824955 11101 raft_consensus.cc:515] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.825899 11070 tablet_bootstrap.cc:492] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.826373 11070 ts_tablet_manager.cc:1403] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.074s	user 0.017s	sys 0.000s
I20260709 13:14:08.827385 10843 leader_election.cc:304] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:08.828178 11073 raft_consensus.cc:2804] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:08.828616 11073 raft_consensus.cc:697] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:08.829383 11073 consensus_queue.cc:237] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.829730 11070 raft_consensus.cc:359] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.830811 11070 raft_consensus.cc:385] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.831097 11070 raft_consensus.cc:740] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.831838 11070 consensus_queue.cc:260] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.834332 11078 raft_consensus.cc:493] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.835140 11078 raft_consensus.cc:515] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.837224 11078 leader_election.cc:290] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.837281 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:08.838346 11027 raft_consensus.cc:2393] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fd3e554dff2d4ac1841f6d8c267b8578 in current term 1: Already voted for candidate d86d8ebbcaff4e76af8e3acb67207878 in this term.
W20260709 13:14:08.837311 10919 leader_election.cc:343] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.838863 10919 leader_election.cc:304] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd3e554dff2d4ac1841f6d8c267b8578; no voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:08.840754 11077 raft_consensus.cc:2749] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:08.841133 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:08.841836 10953 raft_consensus.cc:2468] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:08.845261 10992 leader_election.cc:304] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:08.845284 10918 leader_election.cc:304] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd3e554dff2d4ac1841f6d8c267b8578; no voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:08.846725 11077 raft_consensus.cc:2749] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:08.847045 11078 raft_consensus.cc:2804] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:08.846976 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.847507 11078 raft_consensus.cc:493] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.848385 11101 leader_election.cc:290] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
W20260709 13:14:08.848452 10993 leader_election.cc:343] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.847853 11078 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.849339 11070 ts_tablet_manager.cc:1434] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.003s
I20260709 13:14:08.849916 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:08.850869 11069 ts_tablet_manager.cc:1434] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.002s
I20260709 13:14:08.851867 11069 tablet_bootstrap.cc:492] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.856748 11070 tablet_bootstrap.cc:492] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:08.856832 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.859597 10879 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 1.
I20260709 13:14:08.860323 11078 raft_consensus.cc:515] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.860962 10993 leader_election.cc:304] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:08.863250 11076 raft_consensus.cc:2804] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:08.863422 11027 raft_consensus.cc:2393] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd3e554dff2d4ac1841f6d8c267b8578 in current term 1: Already voted for candidate e17dd92ad52846bd93d4c3237acd28e0 in this term.
I20260709 13:14:08.861325 10775 catalog_manager.cc:5697] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 0 to 1, leader changed from <none> to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:08.862394 10919 leader_election.cc:343] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.865612 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:08.868891 10918 leader_election.cc:304] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd3e554dff2d4ac1841f6d8c267b8578; no voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:08.870980 11068 tablet_bootstrap.cc:492] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.869602 11076 raft_consensus.cc:697] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:08.871850 11078 leader_election.cc:290] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:08.873001 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:08.873545 10953 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:08.875018 10993 leader_election.cc:343] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.872277 11076 consensus_queue.cc:237] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.877516 11069 tablet_bootstrap.cc:654] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.874689 11068 ts_tablet_manager.cc:1403] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.007s
I20260709 13:14:08.880535 11070 tablet_bootstrap.cc:654] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.882141 10953 raft_consensus.cc:2468] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 1.
I20260709 13:14:08.883934 11068 raft_consensus.cc:359] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.884616 11068 raft_consensus.cc:385] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.884656 10992 leader_election.cc:304] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: d86d8ebbcaff4e76af8e3acb67207878
I20260709 13:14:08.885018 11068 raft_consensus.cc:740] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.885946 11108 raft_consensus.cc:2804] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:08.886505 11104 raft_consensus.cc:2749] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:08.886745 11108 raft_consensus.cc:697] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:08.887617 11108 consensus_queue.cc:237] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.885735 11068 consensus_queue.cc:260] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.893632 11068 ts_tablet_manager.cc:1434] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 13:14:08.898955 11068 tablet_bootstrap.cc:492] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.899592 11069 tablet_bootstrap.cc:492] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.900171 11069 ts_tablet_manager.cc:1403] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.000s
I20260709 13:14:08.902972 11069 raft_consensus.cc:359] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.905966 11069 raft_consensus.cc:385] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.906328 11069 raft_consensus.cc:740] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.907244 11068 tablet_bootstrap.cc:654] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.907168 11069 consensus_queue.cc:260] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.909793 11069 ts_tablet_manager.cc:1434] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:14:08.910746 11069 tablet_bootstrap.cc:492] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.912946 11070 tablet_bootstrap.cc:492] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.913141 11068 tablet_bootstrap.cc:492] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.913405 11070 ts_tablet_manager.cc:1403] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.021s
I20260709 13:14:08.913516 11068 ts_tablet_manager.cc:1403] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 13:14:08.916720 11069 tablet_bootstrap.cc:654] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.916512 11070 raft_consensus.cc:359] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.917291 11070 raft_consensus.cc:385] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.917613 11070 raft_consensus.cc:740] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.918380 11070 consensus_queue.cc:260] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.920759 11070 ts_tablet_manager.cc:1434] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:14:08.922842 11068 raft_consensus.cc:359] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.923609 11068 raft_consensus.cc:385] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.923916 11068 raft_consensus.cc:740] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.924599 11068 consensus_queue.cc:260] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.921828 11070 tablet_bootstrap.cc:492] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:08.928504 11069 tablet_bootstrap.cc:492] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:08.929054 11069 ts_tablet_manager.cc:1403] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:14:08.930577 11068 ts_tablet_manager.cc:1434] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:14:08.930681 11104 raft_consensus.cc:493] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.931180 11073 raft_consensus.cc:493] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.931635 11068 tablet_bootstrap.cc:492] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.931756 11073 raft_consensus.cc:515] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.934898 10777 catalog_manager.cc:5697] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.937450 11073 leader_election.cc:290] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.938619 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:08.938254 10777 catalog_manager.cc:5697] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.939018 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:08.939253 11027 raft_consensus.cc:2468] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:08.934366 11070 tablet_bootstrap.cc:654] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.940433 10843 leader_election.cc:304] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:08.941246 11073 raft_consensus.cc:2804] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:14:08.941439 10843 leader_election.cc:343] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.941820 11073 raft_consensus.cc:493] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.936897 11069 raft_consensus.cc:359] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.931212 11104 raft_consensus.cc:515] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.942219 11073 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.942807 11068 tablet_bootstrap.cc:654] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.945082 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bb61541b6b4b6f9d6be0312ba07c81" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:08.945309 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bb61541b6b4b6f9d6be0312ba07c81" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:08.945763 10879 raft_consensus.cc:2468] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 0.
I20260709 13:14:08.945919 11027 raft_consensus.cc:2468] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 0.
I20260709 13:14:08.947220 10918 leader_election.cc:304] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:08.948010 11101 raft_consensus.cc:2804] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:08.948113 11069 raft_consensus.cc:385] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.948444 11101 raft_consensus.cc:493] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.948469 11069 raft_consensus.cc:740] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.948931 11101 raft_consensus.cc:3060] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.949350 11069 consensus_queue.cc:260] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.953969 11073 raft_consensus.cc:515] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.948024 11104 leader_election.cc:290] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.955904 11069 ts_tablet_manager.cc:1434] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:14:08.956789 11069 tablet_bootstrap.cc:492] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:08.956900 11101 raft_consensus.cc:515] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.957254 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:08.959789 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bb61541b6b4b6f9d6be0312ba07c81" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:08.959901 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bb61541b6b4b6f9d6be0312ba07c81" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:08.960346 11101 leader_election.cc:290] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.960467 10879 raft_consensus.cc:3060] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:08.964339 10843 leader_election.cc:343] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:08.960496 11027 raft_consensus.cc:3060] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.971851 11068 tablet_bootstrap.cc:492] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:08.972316 11068 ts_tablet_manager.cc:1403] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.012s
I20260709 13:14:08.974803 11068 raft_consensus.cc:359] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.975467 11068 raft_consensus.cc:385] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.975733 11068 raft_consensus.cc:740] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.976450 11068 consensus_queue.cc:260] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.978025 11027 raft_consensus.cc:2468] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 1.
I20260709 13:14:08.980803 11070 tablet_bootstrap.cc:492] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:08.980856 10879 raft_consensus.cc:2468] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 1.
I20260709 13:14:08.981340 11070 ts_tablet_manager.cc:1403] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.010s
I20260709 13:14:08.982035 11068 ts_tablet_manager.cc:1434] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:14:08.982157 10919 leader_election.cc:304] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:08.983165 11068 tablet_bootstrap.cc:492] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:08.983073 11101 raft_consensus.cc:2804] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:08.983737 11101 raft_consensus.cc:697] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 LEADER]: Becoming Leader. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:08.984511 11101 consensus_queue.cc:237] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.984860 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:08.985438 11026 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:08.987152 11073 leader_election.cc:290] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:08.988610 11070 raft_consensus.cc:359] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.989262 11070 raft_consensus.cc:385] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:08.989410 11068 tablet_bootstrap.cc:654] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.989557 11070 raft_consensus.cc:740] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:08.990361 11070 consensus_queue.cc:260] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.992378 11070 ts_tablet_manager.cc:1434] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:14:08.993464 11069 tablet_bootstrap.cc:654] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:08.995932 11026 raft_consensus.cc:2468] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 1.
I20260709 13:14:08.997237 10843 leader_election.cc:304] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:08.998397 11073 raft_consensus.cc:2804] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:08.998858 11073 raft_consensus.cc:697] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:08.999679 11073 consensus_queue.cc:237] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:08.993633 11070 tablet_bootstrap.cc:492] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:09.009069 11070 tablet_bootstrap.cc:654] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:09.011205 11069 tablet_bootstrap.cc:492] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:09.011763 11069 ts_tablet_manager.cc:1403] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.004s
I20260709 13:14:09.014853 11068 tablet_bootstrap.cc:492] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:09.015278 11068 ts_tablet_manager.cc:1403] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.032s	user 0.021s	sys 0.004s
I20260709 13:14:09.017735 11068 raft_consensus.cc:359] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.018371 11068 raft_consensus.cc:385] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:09.018632 11068 raft_consensus.cc:740] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:09.018795 11069 raft_consensus.cc:359] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.019275 11068 consensus_queue.cc:260] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.019555 11069 raft_consensus.cc:385] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:09.019973 11069 raft_consensus.cc:740] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:09.020184 10775 catalog_manager.cc:5697] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 0 to 1, leader changed from <none> to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.021134 11068 ts_tablet_manager.cc:1434] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:09.019562 10776 catalog_manager.cc:5697] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.021088 11069 consensus_queue.cc:260] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.022162 11068 tablet_bootstrap.cc:492] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578: Bootstrap starting.
I20260709 13:14:09.023128 11069 ts_tablet_manager.cc:1434] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.004s
I20260709 13:14:09.025010 11070 tablet_bootstrap.cc:492] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:09.025425 11070 ts_tablet_manager.cc:1403] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 13:14:09.027657 11069 tablet_bootstrap.cc:492] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: Bootstrap starting.
I20260709 13:14:09.027931 11068 tablet_bootstrap.cc:654] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:09.027930 11070 raft_consensus.cc:359] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.028667 11070 raft_consensus.cc:385] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:09.028910 11070 raft_consensus.cc:740] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:09.029690 11070 consensus_queue.cc:260] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.033432 11069 tablet_bootstrap.cc:654] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:09.034649 11073 raft_consensus.cc:493] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.035092 11073 raft_consensus.cc:515] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.037173 11073 leader_election.cc:290] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.038983 11068 tablet_bootstrap.cc:492] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578: No bootstrap required, opened a new log
I20260709 13:14:09.039139 11070 ts_tablet_manager.cc:1434] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:14:09.039428 11068 ts_tablet_manager.cc:1403] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 13:14:09.041450 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:09.041927 11068 raft_consensus.cc:359] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.042407 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:09.042572 11068 raft_consensus.cc:385] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:09.042935 11068 raft_consensus.cc:740] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Initialized, Role: FOLLOWER
I20260709 13:14:09.043131 10953 raft_consensus.cc:2393] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d86d8ebbcaff4e76af8e3acb67207878 in current term 1: Already voted for candidate e17dd92ad52846bd93d4c3237acd28e0 in this term.
I20260709 13:14:09.043730 11068 consensus_queue.cc:260] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.044457 10843 leader_election.cc:304] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878; no voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:09.045480 11073 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.045630 11068 ts_tablet_manager.cc:1434] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:14:09.051152 11070 tablet_bootstrap.cc:492] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0: Bootstrap starting.
I20260709 13:14:09.051087 11073 raft_consensus.cc:2749] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:09.052973 11069 tablet_bootstrap.cc:492] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: No bootstrap required, opened a new log
I20260709 13:14:09.053342 11069 ts_tablet_manager.cc:1403] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.006s
I20260709 13:14:09.055850 11069 raft_consensus.cc:359] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.056200 11070 tablet_bootstrap.cc:654] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:09.056376 11069 raft_consensus.cc:385] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:09.056636 11069 raft_consensus.cc:740] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Initialized, Role: FOLLOWER
I20260709 13:14:09.057160 11069 consensus_queue.cc:260] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.058552 11069 ts_tablet_manager.cc:1434] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:14:09.060649 11070 tablet_bootstrap.cc:492] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0: No bootstrap required, opened a new log
I20260709 13:14:09.061070 11070 ts_tablet_manager.cc:1403] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0: Time spent bootstrapping tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 13:14:09.063567 11070 raft_consensus.cc:359] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.064148 11070 raft_consensus.cc:385] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:09.064365 11070 raft_consensus.cc:740] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Initialized, Role: FOLLOWER
I20260709 13:14:09.064831 11070 consensus_queue.cc:260] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.066316 11070 ts_tablet_manager.cc:1434] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:09.083870 11073 raft_consensus.cc:493] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.084292 11073 raft_consensus.cc:515] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.084728 11108 raft_consensus.cc:493] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.085139 11108 raft_consensus.cc:515] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.085997 11073 leader_election.cc:290] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.086717 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:09.086824 11108 leader_election.cc:290] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:09.087052 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:09.087302 10953 raft_consensus.cc:2468] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:09.087636 11026 raft_consensus.cc:2468] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:09.087627 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:09.088232 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:09.088377 10879 raft_consensus.cc:2468] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:09.088450 10843 leader_election.cc:304] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:09.088755 10954 raft_consensus.cc:2468] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:09.089071 11073 raft_consensus.cc:2804] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:09.089404 11073 raft_consensus.cc:493] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.089564 10993 leader_election.cc:304] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:09.089756 11073 raft_consensus.cc:3060] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.090358 11108 raft_consensus.cc:2804] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:09.090690 11108 raft_consensus.cc:493] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.090997 11108 raft_consensus.cc:3060] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.096189 11073 raft_consensus.cc:515] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.097149 11108 raft_consensus.cc:515] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.098057 11073 leader_election.cc:290] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.098740 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:09.099143 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:09.099267 10954 raft_consensus.cc:3060] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.099824 11026 raft_consensus.cc:2393] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d86d8ebbcaff4e76af8e3acb67207878 in current term 1: Already voted for candidate e17dd92ad52846bd93d4c3237acd28e0 in this term.
I20260709 13:14:09.099948 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:09.099845 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:09.100852 10879 raft_consensus.cc:2393] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e17dd92ad52846bd93d4c3237acd28e0 in current term 1: Already voted for candidate d86d8ebbcaff4e76af8e3acb67207878 in this term.
I20260709 13:14:09.101054 11108 leader_election.cc:290] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:09.101786 10993 leader_election.cc:304] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0; no voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:09.102330 11108 raft_consensus.cc:2749] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:09.104207 10954 raft_consensus.cc:2468] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 1.
I20260709 13:14:09.105077 10843 leader_election.cc:304] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:09.105643 11073 raft_consensus.cc:2804] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:09.106001 11073 raft_consensus.cc:697] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:09.106726 11073 consensus_queue.cc:237] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.112797 11108 raft_consensus.cc:493] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.113229 11108 raft_consensus.cc:515] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.114454 10775 catalog_manager.cc:5697] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 0 to 1, leader changed from <none> to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.115235 11108 leader_election.cc:290] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:09.115926 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:09.116309 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:09.116434 10879 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:09.116911 10954 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:09.117661 10993 leader_election.cc:304] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:09.118320 11108 raft_consensus.cc:2804] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:09.118646 11108 raft_consensus.cc:493] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.118928 11108 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.124361 11108 raft_consensus.cc:515] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.125850 11108 leader_election.cc:290] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:09.126442 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:09.126650 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:09.126935 10879 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.127130 10954 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.131564 10879 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 1.
I20260709 13:14:09.131608 10954 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 1.
I20260709 13:14:09.132500 10993 leader_election.cc:304] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:09.133426 11108 raft_consensus.cc:2804] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:09.133773 11108 raft_consensus.cc:697] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:09.134445 11108 consensus_queue.cc:237] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.138589 11108 consensus_queue.cc:1048] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.142549 10775 catalog_manager.cc:5697] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.154479 11076 consensus_queue.cc:1048] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:09.225318 11101 raft_consensus.cc:493] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.225827 11101 raft_consensus.cc:515] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.227617 11101 leader_election.cc:290] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.228331 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:09.228652 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:09.228977 10879 raft_consensus.cc:2393] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd3e554dff2d4ac1841f6d8c267b8578 in current term 1: Already voted for candidate e17dd92ad52846bd93d4c3237acd28e0 in this term.
I20260709 13:14:09.229938 10919 leader_election.cc:304] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd3e554dff2d4ac1841f6d8c267b8578; no voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:09.230516 11101 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.234787 11101 raft_consensus.cc:2749] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:09.281368 11076 consensus_queue.cc:1048] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.287079 11108 raft_consensus.cc:493] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.287566 11108 raft_consensus.cc:515] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.289425 11108 leader_election.cc:290] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:09.290475 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3658791733f84b198f0385c5dac7cca2" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:09.291105 10953 raft_consensus.cc:2468] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:09.291714 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3658791733f84b198f0385c5dac7cca2" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:09.292274 10876 raft_consensus.cc:2468] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 0.
I20260709 13:14:09.293684 10992 leader_election.cc:304] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:09.294718 11078 raft_consensus.cc:2804] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:09.294960 11134 consensus_queue.cc:1048] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:09.296556 11073 consensus_queue.cc:1048] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.295061 11078 raft_consensus.cc:493] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.297869 11078 raft_consensus.cc:3060] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.300309 11076 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.312950 11078 raft_consensus.cc:515] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.316247 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3658791733f84b198f0385c5dac7cca2" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:09.316382 10953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3658791733f84b198f0385c5dac7cca2" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:09.317025 10879 raft_consensus.cc:3060] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.317052 10953 raft_consensus.cc:3060] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.318010 11078 leader_election.cc:290] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:09.326766 10879 raft_consensus.cc:2468] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 1.
I20260709 13:14:09.331578 10993 leader_election.cc:304] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:09.332647 11108 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.333137 11076 raft_consensus.cc:2804] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:09.333619 11076 raft_consensus.cc:697] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:09.334395 11076 consensus_queue.cc:237] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.336683 10953 raft_consensus.cc:2468] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 1.
I20260709 13:14:09.342607 11095 consensus_queue.cc:1048] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.354025 10776 catalog_manager.cc:5697] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.413167 11095 raft_consensus.cc:493] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.413568 11095 raft_consensus.cc:515] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.415050 11095 leader_election.cc:290] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.415840 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" is_pre_election: true
I20260709 13:14:09.416189 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:09.416383 10954 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:09.416752 11026 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 0.
I20260709 13:14:09.417299 10843 leader_election.cc:304] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:09.417873 11095 raft_consensus.cc:2804] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:09.418126 11095 raft_consensus.cc:493] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.418372 11095 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.422987 11095 raft_consensus.cc:515] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.423789 11101 raft_consensus.cc:493] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.424170 11101 raft_consensus.cc:515] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.424576 11095 leader_election.cc:290] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.425403 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:09.426031 11101 leader_election.cc:290] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.426323 10954 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.426680 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:09.427186 11026 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.428012 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878" is_pre_election: true
I20260709 13:14:09.429006 11101 consensus_queue.cc:1048] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.429207 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0" is_pre_election: true
I20260709 13:14:09.429937 11027 raft_consensus.cc:2393] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fd3e554dff2d4ac1841f6d8c267b8578 in current term 1: Already voted for candidate d86d8ebbcaff4e76af8e3acb67207878 in this term.
I20260709 13:14:09.433037 10954 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 1.
I20260709 13:14:09.436645 11026 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 1.
I20260709 13:14:09.438310 10843 leader_election.cc:304] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:09.432600 10918 leader_election.cc:304] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd3e554dff2d4ac1841f6d8c267b8578; no voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:09.439131 11095 raft_consensus.cc:2804] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:09.439320 11101 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:09.439678 11095 raft_consensus.cc:697] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:09.440672 11095 consensus_queue.cc:237] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.447057 11101 raft_consensus.cc:2749] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:09.451869 11101 consensus_queue.cc:1048] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.462631 10776 catalog_manager.cc:5697] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 0 to 1, leader changed from <none> to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.523206 11076 consensus_queue.cc:1048] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.532871 11134 consensus_queue.cc:1048] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.566964 11095 consensus_queue.cc:1048] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.577536 11149 consensus_queue.cc:1048] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.654809 11026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
mode: GRACEFUL
new_leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:35518
I20260709 13:14:09.655297 11026 raft_consensus.cc:606] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Received request to transfer leadership to TS d86d8ebbcaff4e76af8e3acb67207878
I20260709 13:14:09.664301 11026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
mode: GRACEFUL
new_leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:35518
I20260709 13:14:09.664726 11026 raft_consensus.cc:606] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Received request to transfer leadership to TS fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:09.668530 11026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
mode: GRACEFUL
new_leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:35518
I20260709 13:14:09.668922 11026 raft_consensus.cc:606] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Received request to transfer leadership to TS fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:09.672844 11026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
mode: GRACEFUL
new_leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:35518
I20260709 13:14:09.673280 11026 raft_consensus.cc:606] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Received request to transfer leadership to TS fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:09.689209 11108 raft_consensus.cc:993] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0: : Instructing follower fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:09.689656 11134 raft_consensus.cc:1081] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Signalling peer fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:09.690899 10954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:14:09.691318 10954 raft_consensus.cc:493] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:09.691689 10954 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.695812 10954 raft_consensus.cc:515] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.697273 10954 leader_election.cc:290] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.697950 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:09.698269 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:09.698482 10876 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.698760 11026 raft_consensus.cc:3055] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:09.699011 11026 raft_consensus.cc:740] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:09.699649 11026 consensus_queue.cc:260] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.700484 11026 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.702896 10876 raft_consensus.cc:2468] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:09.703851 10919 leader_election.cc:304] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:09.704452 11144 raft_consensus.cc:2804] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.704912 11144 raft_consensus.cc:697] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Becoming Leader. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:09.704902 11026 raft_consensus.cc:2468] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:09.705844 11144 consensus_queue.cc:237] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.712189 10777 catalog_manager.cc:5697] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 reported cstate change: term changed from 1 to 2, leader changed from e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67) to fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.714474 11108 raft_consensus.cc:993] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0: : Instructing follower d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:09.714892 11134 raft_consensus.cc:1081] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Signalling peer d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:09.718039 10876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:47478
I20260709 13:14:09.718483 10876 raft_consensus.cc:493] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:09.718755 10876 raft_consensus.cc:3060] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.723507 11149 consensus_queue.cc:1048] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.724020 10876 raft_consensus.cc:515] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.725710 10876 leader_election.cc:290] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 2 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.726604 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:09.727129 10954 raft_consensus.cc:3060] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.727115 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aada48f8534745c19e54a404fc4a3037" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:09.727917 11027 raft_consensus.cc:3055] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:09.728291 11027 raft_consensus.cc:740] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:09.729045 11027 consensus_queue.cc:260] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.730129 11027 raft_consensus.cc:3060] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.735770 11027 raft_consensus.cc:2468] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 2.
I20260709 13:14:09.738462 10954 raft_consensus.cc:2468] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 2.
I20260709 13:14:09.740100 10843 leader_election.cc:304] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:09.740887 11154 raft_consensus.cc:2804] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.741310 11154 raft_consensus.cc:697] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:09.742202 11073 consensus_queue.cc:1048] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:09.742190 11154 consensus_queue.cc:237] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.750383 10777 catalog_manager.cc:5697] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 1 to 2, leader changed from e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67) to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:09.849340 11073 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.859081 11154 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.917975 11134 consensus_queue.cc:1048] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.927446 11076 consensus_queue.cc:1048] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:09.955821 11108 raft_consensus.cc:993] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0: : Instructing follower fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:09.956285 11076 raft_consensus.cc:1081] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Signalling peer fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:09.957615 10954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:14:09.958065 10954 raft_consensus.cc:493] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:09.958413 10954 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.962787 10954 raft_consensus.cc:515] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.964499 10954 leader_election.cc:290] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:09.965220 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:09.965464 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:09.965688 10876 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.965955 11027 raft_consensus.cc:3055] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:09.966203 11027 raft_consensus.cc:740] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:09.966823 11027 consensus_queue.cc:260] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.967700 11027 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:09.970244 10876 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:09.971256 10919 leader_election.cc:304] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:09.972018 11144 raft_consensus.cc:2804] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:09.972332 11027 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:09.972399 11144 raft_consensus.cc:697] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Becoming Leader. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:09.973246 11144 consensus_queue.cc:237] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:09.979082 10777 catalog_manager.cc:5697] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 reported cstate change: term changed from 1 to 2, leader changed from e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67) to fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.041587 11108 raft_consensus.cc:993] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0: : Instructing follower fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:10.041952 11076 raft_consensus.cc:1081] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Signalling peer fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:10.043193 10954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:14:10.043623 10954 raft_consensus.cc:493] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:10.043870 10954 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:10.048127 10954 raft_consensus.cc:515] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:10.049880 10954 leader_election.cc:290] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:10.050884 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:10.051473 10876 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:10.051898 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:10.052438 11027 raft_consensus.cc:3055] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:10.052757 11027 raft_consensus.cc:740] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:10.053435 11027 consensus_queue.cc:260] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:10.054313 11027 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:10.056510 10876 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:10.057530 10919 leader_election.cc:304] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:10.058295 11162 raft_consensus.cc:2804] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:10.058642 11162 raft_consensus.cc:697] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Becoming Leader. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:10.059165 11027 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:10.059366 11162 consensus_queue.cc:237] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:10.064947 10777 catalog_manager.cc:5697] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 reported cstate change: term changed from 1 to 2, leader changed from e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67) to fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.117029 10876 raft_consensus.cc:1275] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.118191 11162 consensus_queue.cc:1048] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.126578 11027 raft_consensus.cc:1275] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.127918 11144 consensus_queue.cc:1048] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.196084 11027 raft_consensus.cc:1275] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.197233 11156 consensus_queue.cc:1048] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.207576 10954 raft_consensus.cc:1275] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.209018 11073 consensus_queue.cc:1048] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.405900 11027 raft_consensus.cc:1275] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.407049 11144 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.415356 10876 raft_consensus.cc:1275] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.417738 11144 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.503778 10876 raft_consensus.cc:1275] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.504909 11144 consensus_queue.cc:1048] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:10.511803 11027 raft_consensus.cc:1275] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:10.513516 11144 consensus_queue.cc:1048] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:12.708379 10257 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:14:12.709156 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:12.724525 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
mode: GRACEFUL
new_leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:57700
I20260709 13:14:12.725018 10876 raft_consensus.cc:606] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Received request to transfer leadership to TS e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:12.727277 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34941245c64f41ffb49311a76630551a"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
mode: GRACEFUL
new_leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:57700
I20260709 13:14:12.727763 10876 raft_consensus.cc:606] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Received request to transfer leadership to TS e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:12.728767 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:14:12.729215 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:12.817654 11171 raft_consensus.cc:993] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: : Instructing follower e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:12.818156 11171 raft_consensus.cc:1081] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Signalling peer e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:12.819871 11026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:41548
I20260709 13:14:12.820444 11026 raft_consensus.cc:493] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:12.820791 11026 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:12.825587 11026 raft_consensus.cc:515] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:12.827320 11026 leader_election.cc:290] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 2 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:12.828210 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:12.828830 10876 raft_consensus.cc:3055] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:12.829005 10952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:12.829155 10876 raft_consensus.cc:740] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:12.829658 10952 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:12.830119 10876 consensus_queue.cc:260] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:12.831348 10876 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:12.835968 10952 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 2.
I20260709 13:14:12.836745 10876 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 2.
I20260709 13:14:12.837244 10992 leader_election.cc:304] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:12.838117 11169 raft_consensus.cc:2804] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:12.839043 11169 raft_consensus.cc:697] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:12.839783 11169 consensus_queue.cc:237] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:12.847828 10776 catalog_manager.cc:5697] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 1 to 2, leader changed from d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65) to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:12.882346 11171 raft_consensus.cc:993] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878: : Instructing follower e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:12.882792 11171 raft_consensus.cc:1081] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Signalling peer e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:12.884545 11026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34941245c64f41ffb49311a76630551a"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:41548
I20260709 13:14:12.885142 11026 raft_consensus.cc:493] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:12.885540 11026 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:12.890291 11026 raft_consensus.cc:515] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:12.892072 11026 leader_election.cc:290] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 2 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:12.893007 10952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:12.892956 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:12.893627 10952 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:12.893919 10876 raft_consensus.cc:3055] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:12.894253 10876 raft_consensus.cc:740] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:12.895051 10876 consensus_queue.cc:260] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:12.896317 10876 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:12.900580 10952 raft_consensus.cc:2468] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 2.
I20260709 13:14:12.901785 10992 leader_election.cc:304] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:12.902457 11169 raft_consensus.cc:2804] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:12.902658 10876 raft_consensus.cc:2468] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 2.
I20260709 13:14:12.903056 11169 raft_consensus.cc:697] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:12.903901 11169 consensus_queue.cc:237] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:12.910511 10776 catalog_manager.cc:5697] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 1 to 2, leader changed from d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65) to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:13.304368 10876 raft_consensus.cc:1275] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:13.305447 11169 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:13.314219 10952 raft_consensus.cc:1275] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:13.315613 11169 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:13.510972 10876 raft_consensus.cc:1275] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:13.511852 10954 raft_consensus.cc:1275] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:13.512164 11184 consensus_queue.cc:1048] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:13.512971 11175 consensus_queue.cc:1048] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:13.729945 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:13.742698 11027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
mode: GRACEFUL
new_leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:35518
I20260709 13:14:13.743165 11027 raft_consensus.cc:606] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Received request to transfer leadership to TS d86d8ebbcaff4e76af8e3acb67207878
I20260709 13:14:13.751776 11027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34941245c64f41ffb49311a76630551a"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
mode: GRACEFUL
new_leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:35518
I20260709 13:14:13.752226 11027 raft_consensus.cc:606] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Received request to transfer leadership to TS d86d8ebbcaff4e76af8e3acb67207878
I20260709 13:14:13.767831 10954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
mode: GRACEFUL
new_leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:53224
I20260709 13:14:13.768293 10954 raft_consensus.cc:606] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Received request to transfer leadership to TS d86d8ebbcaff4e76af8e3acb67207878
I20260709 13:14:13.772691 10954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
mode: GRACEFUL
new_leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:53224
I20260709 13:14:13.773162 10954 raft_consensus.cc:606] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Received request to transfer leadership to TS d86d8ebbcaff4e76af8e3acb67207878
I20260709 13:14:13.777421 10954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
mode: GRACEFUL
new_leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:53224
I20260709 13:14:13.777832 10954 raft_consensus.cc:606] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Received request to transfer leadership to TS d86d8ebbcaff4e76af8e3acb67207878
I20260709 13:14:13.827672 11162 raft_consensus.cc:993] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578: : Instructing follower d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:13.828068 11193 raft_consensus.cc:1081] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Signalling peer d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:13.829474 10876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:47484
I20260709 13:14:13.829983 10876 raft_consensus.cc:493] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:13.830261 10876 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.835481 10876 raft_consensus.cc:515] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.836962 10876 leader_election.cc:290] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:13.837817 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:13.837992 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:13.838384 10954 raft_consensus.cc:3055] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:13.838498 11027 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.838687 10954 raft_consensus.cc:740] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:13.839361 10954 consensus_queue.cc:260] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.840479 10954 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.842317 11190 raft_consensus.cc:993] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0: : Instructing follower d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:13.842852 11175 raft_consensus.cc:1081] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Signalling peer d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:13.844515 11027 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:13.844544 10876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:47478
I20260709 13:14:13.845183 10876 raft_consensus.cc:493] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:13.845572 10843 leader_election.cc:304] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:13.845566 10876 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.846150 10954 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:13.846309 11171 raft_consensus.cc:2804] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:13.847576 11171 raft_consensus.cc:697] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:13.848273 11171 consensus_queue.cc:237] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.850463 10876 raft_consensus.cc:515] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.852170 10876 leader_election.cc:290] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:13.853434 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:13.853420 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:13.854103 11027 raft_consensus.cc:3055] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:13.854421 11027 raft_consensus.cc:740] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:13.855568 10954 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.855219 11027 consensus_queue.cc:260] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.855633 10777 catalog_manager.cc:5697] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 2 to 3, leader changed from fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66) to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:13.856520 11027 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.859926 10954 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:13.861002 10843 leader_election.cc:304] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:13.861761 11171 raft_consensus.cc:2804] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:13.862242 11171 raft_consensus.cc:697] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:13.863090 11171 consensus_queue.cc:237] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.864370 11027 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:13.871536 10777 catalog_manager.cc:5697] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 2 to 3, leader changed from e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67) to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:13.879745 11162 raft_consensus.cc:993] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578: : Instructing follower d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:13.880118 11193 raft_consensus.cc:1081] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Signalling peer d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:13.881361 10876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:47484
I20260709 13:14:13.881764 10876 raft_consensus.cc:493] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:13.881986 10876 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.885737 10876 raft_consensus.cc:515] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.886996 10876 leader_election.cc:290] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:13.887868 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:13.888080 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93e7d762c4d04bc0a93f865c0e4fffa9" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:13.888296 10954 raft_consensus.cc:3055] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:13.888545 10954 raft_consensus.cc:740] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:13.888556 11027 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.889145 10954 consensus_queue.cc:260] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.890201 10954 raft_consensus.cc:3060] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.894858 11027 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:13.896135 10843 leader_election.cc:304] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:13.896169 10954 raft_consensus.cc:2468] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:13.896906 11171 raft_consensus.cc:2804] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:13.897400 11171 raft_consensus.cc:697] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:13.898204 11171 consensus_queue.cc:237] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:13.904428 10777 catalog_manager.cc:5697] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 2 to 3, leader changed from fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66) to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:14.005748 11190 raft_consensus.cc:993] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0: : Instructing follower d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:14.006124 11175 raft_consensus.cc:1081] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Signalling peer d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:14.007468 10876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34941245c64f41ffb49311a76630551a"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:47478
I20260709 13:14:14.007879 10876 raft_consensus.cc:493] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:14.008136 10876 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:14.012033 10876 raft_consensus.cc:515] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:14.013489 10876 leader_election.cc:290] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:14.014340 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:14.014622 11027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34941245c64f41ffb49311a76630551a" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:14.014796 10954 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:14.015090 11027 raft_consensus.cc:3055] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:14.015378 11027 raft_consensus.cc:740] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:14.016064 11027 consensus_queue.cc:260] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:14.016956 11027 raft_consensus.cc:3060] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:14.018959 10954 raft_consensus.cc:2468] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:14.019912 10843 leader_election.cc:304] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:14.020602 11171 raft_consensus.cc:2804] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:14.020988 11171 raft_consensus.cc:697] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:14.021538 11027 raft_consensus.cc:2468] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:14.021783 11171 consensus_queue.cc:237] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:14.028515 10777 catalog_manager.cc:5697] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 2 to 3, leader changed from e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67) to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:14.226027 11193 raft_consensus.cc:993] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578: : Instructing follower d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:14.226481 11162 raft_consensus.cc:1081] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Signalling peer d86d8ebbcaff4e76af8e3acb67207878 to start an election
I20260709 13:14:14.227870 10876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
 from {username='slave'} at 127.0.0.1:47484
I20260709 13:14:14.228241 10876 raft_consensus.cc:493] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:14.228454 10876 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:14.232638 10876 raft_consensus.cc:515] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:14.233903 10876 leader_election.cc:290] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:14.234772 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:14.235100 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752c0c8fd42c0be6146dc0a7d1dbf" candidate_uuid: "d86d8ebbcaff4e76af8e3acb67207878" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:14.235435 10954 raft_consensus.cc:3055] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:14.235663 11026 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:14.235777 10954 raft_consensus.cc:740] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:14.236450 10954 consensus_queue.cc:260] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:14.237205 10954 raft_consensus.cc:3060] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:14.240229 11026 raft_consensus.cc:2468] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:14.241338 10843 leader_election.cc:304] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:14.241405 10954 raft_consensus.cc:2468] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86d8ebbcaff4e76af8e3acb67207878 in term 3.
I20260709 13:14:14.242014 11194 raft_consensus.cc:2804] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:14.242444 11194 raft_consensus.cc:697] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Becoming Leader. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:14.243155 11194 consensus_queue.cc:237] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:14.249750 10777 catalog_manager.cc:5697] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 reported cstate change: term changed from 2 to 3, leader changed from fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66) to d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "d86d8ebbcaff4e76af8e3acb67207878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:14.314734 11026 raft_consensus.cc:1275] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.316015 11171 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.321871 10954 raft_consensus.cc:1275] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.326053 10952 raft_consensus.cc:1275] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.327477 11194 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.328537 11204 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.333364 11026 raft_consensus.cc:1275] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.334838 11206 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.388887 11026 raft_consensus.cc:1275] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.390090 11204 consensus_queue.cc:1048] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.399816 10952 raft_consensus.cc:1275] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.401180 11204 consensus_queue.cc:1048] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.431798 11026 raft_consensus.cc:1275] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.432932 11171 consensus_queue.cc:1048] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.439877 10952 raft_consensus.cc:1275] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.441470 11206 consensus_queue.cc:1048] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.763689 11026 raft_consensus.cc:1275] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.764813 11206 consensus_queue.cc:1048] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:14.772377 10952 raft_consensus.cc:1275] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Refusing update from remote peer d86d8ebbcaff4e76af8e3acb67207878: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:14.773957 11204 consensus_queue.cc:1048] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:16.784354 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:16.793049 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
mode: GRACEFUL
new_leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:57700
I20260709 13:14:16.793792 10876 raft_consensus.cc:606] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Received request to transfer leadership to TS e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:16.796293 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
mode: GRACEFUL
new_leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:57700
I20260709 13:14:16.796753 10876 raft_consensus.cc:606] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Received request to transfer leadership to TS e17dd92ad52846bd93d4c3237acd28e0
I20260709 13:14:16.798870 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
mode: GRACEFUL
new_leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:57700
I20260709 13:14:16.799299 10876 raft_consensus.cc:606] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Received request to transfer leadership to TS fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:16.801353 10876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a"
dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
mode: GRACEFUL
new_leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:57700
I20260709 13:14:16.801779 10876 raft_consensus.cc:606] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Received request to transfer leadership to TS fd3e554dff2d4ac1841f6d8c267b8578
I20260709 13:14:16.802709 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:14:16.803126 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:16.817764 11216 raft_consensus.cc:993] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878: : Instructing follower fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:16.818171 11216 raft_consensus.cc:1081] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Signalling peer fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:16.819626 10952 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:34986
I20260709 13:14:16.820079 10952 raft_consensus.cc:493] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:16.820399 10952 raft_consensus.cc:3060] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:16.825946 10952 raft_consensus.cc:515] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:16.828205 10952 leader_election.cc:290] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:16.829042 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:16.829437 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7946fded4dfd4022b386d9485b8ca5ce" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:16.829782 10876 raft_consensus.cc:3055] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:16.829953 11026 raft_consensus.cc:3060] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:16.830140 10876 raft_consensus.cc:740] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:16.830875 10876 consensus_queue.cc:260] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:16.831852 10876 raft_consensus.cc:3060] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:16.834651 11026 raft_consensus.cc:2468] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:16.835940 10918 leader_election.cc:304] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:16.836624 10876 raft_consensus.cc:2468] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 2.
I20260709 13:14:16.836745 11217 raft_consensus.cc:2804] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:16.838519 11217 raft_consensus.cc:697] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Becoming Leader. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:16.840117 11217 consensus_queue.cc:237] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:16.846621 10776 catalog_manager.cc:5697] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 reported cstate change: term changed from 1 to 2, leader changed from d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65) to fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:17.175858 11218 raft_consensus.cc:993] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878: : Instructing follower e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:17.176266 11216 raft_consensus.cc:1081] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Signalling peer e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:17.177582 11026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:41548
I20260709 13:14:17.177968 11026 raft_consensus.cc:493] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:17.178190 11026 raft_consensus.cc:3060] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:17.182080 11026 raft_consensus.cc:515] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.183427 11026 leader_election.cc:290] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 2 election: Requested vote from peers fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223), d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757)
I20260709 13:14:17.184306 10952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:17.184386 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e64608e7c94b7caa8664d7aed15b50" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:17.184875 10952 raft_consensus.cc:3060] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:17.184931 10876 raft_consensus.cc:3055] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:17.185213 10876 raft_consensus.cc:740] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:17.185724 10876 consensus_queue.cc:260] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.186480 10876 raft_consensus.cc:3060] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:17.189586 10952 raft_consensus.cc:2468] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 2.
I20260709 13:14:17.190864 10992 leader_election.cc:304] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:17.191047 10876 raft_consensus.cc:2468] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 2.
I20260709 13:14:17.191713 11175 raft_consensus.cc:2804] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:17.193053 11175 raft_consensus.cc:697] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:17.193775 11175 consensus_queue.cc:237] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.200277 10776 catalog_manager.cc:5697] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 1 to 2, leader changed from d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65) to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.262318 10876 raft_consensus.cc:1275] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:17.263473 11217 consensus_queue.cc:1048] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:17.273017 11026 raft_consensus.cc:1275] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:17.274667 11217 consensus_queue.cc:1048] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:17.286018 11218 raft_consensus.cc:993] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: : Instructing follower e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:17.286432 11216 raft_consensus.cc:1081] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Signalling peer e17dd92ad52846bd93d4c3237acd28e0 to start an election
I20260709 13:14:17.287889 11026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc"
dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
 from {username='slave'} at 127.0.0.1:41548
I20260709 13:14:17.288391 11026 raft_consensus.cc:493] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:17.288724 11026 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:17.290167 11216 raft_consensus.cc:993] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878: : Instructing follower fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:17.290632 11218 raft_consensus.cc:1081] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Signalling peer fd3e554dff2d4ac1841f6d8c267b8578 to start an election
I20260709 13:14:17.293157 10952 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a"
dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
 from {username='slave'} at 127.0.0.1:34986
I20260709 13:14:17.293617 10952 raft_consensus.cc:493] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:17.293896 10952 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:17.295100 11026 raft_consensus.cc:515] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.296661 11026 leader_election.cc:290] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 4 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66:34223)
I20260709 13:14:17.297420 10876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:17.297753 10954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0658ee47d6e4413e9f207532ea4999cc" candidate_uuid: "e17dd92ad52846bd93d4c3237acd28e0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fd3e554dff2d4ac1841f6d8c267b8578"
I20260709 13:14:17.297991 10876 raft_consensus.cc:3055] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:17.298305 10876 raft_consensus.cc:740] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:17.298532 10954 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:17.299391 10876 consensus_queue.cc:260] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.300212 10952 raft_consensus.cc:515] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.300566 10876 raft_consensus.cc:3060] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:17.302645 10952 leader_election.cc:290] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 4 election: Requested vote from peers d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757), e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67:44365)
I20260709 13:14:17.303674 10879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d86d8ebbcaff4e76af8e3acb67207878"
I20260709 13:14:17.304211 10954 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 4.
I20260709 13:14:17.304260 10879 raft_consensus.cc:3055] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:17.304697 10879 raft_consensus.cc:740] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d86d8ebbcaff4e76af8e3acb67207878, State: Running, Role: LEADER
I20260709 13:14:17.305292 11026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8634103b437b49a8b0d8724e44da474a" candidate_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e17dd92ad52846bd93d4c3237acd28e0"
I20260709 13:14:17.305816 11026 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:17.305531 10879 consensus_queue.cc:260] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.305972 10876 raft_consensus.cc:2468] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e17dd92ad52846bd93d4c3237acd28e0 in term 4.
I20260709 13:14:17.306860 10879 raft_consensus.cc:3060] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:17.307367 10993 leader_election.cc:304] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86d8ebbcaff4e76af8e3acb67207878, e17dd92ad52846bd93d4c3237acd28e0; no voters: 
I20260709 13:14:17.308202 11175 raft_consensus.cc:2804] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:17.308643 11175 raft_consensus.cc:697] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 4 LEADER]: Becoming Leader. State: Replica: e17dd92ad52846bd93d4c3237acd28e0, State: Running, Role: LEADER
I20260709 13:14:17.309476 11175 consensus_queue.cc:237] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.312156 11026 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 4.
I20260709 13:14:17.313378 10918 leader_election.cc:304] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e17dd92ad52846bd93d4c3237acd28e0, fd3e554dff2d4ac1841f6d8c267b8578; no voters: 
I20260709 13:14:17.314159 11217 raft_consensus.cc:2804] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:17.314846 11217 raft_consensus.cc:697] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 LEADER]: Becoming Leader. State: Replica: fd3e554dff2d4ac1841f6d8c267b8578, State: Running, Role: LEADER
I20260709 13:14:17.315788 11217 consensus_queue.cc:237] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } }
I20260709 13:14:17.321246 10879 raft_consensus.cc:2468] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd3e554dff2d4ac1841f6d8c267b8578 in term 4.
I20260709 13:14:17.324056 10777 catalog_manager.cc:5697] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 reported cstate change: term changed from 3 to 4, leader changed from d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65) to e17dd92ad52846bd93d4c3237acd28e0 (127.10.4.67). New cstate: current_term: 4 leader_uuid: "e17dd92ad52846bd93d4c3237acd28e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.324752 10776 catalog_manager.cc:5697] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 reported cstate change: term changed from 3 to 4, leader changed from d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65) to fd3e554dff2d4ac1841f6d8c267b8578 (127.10.4.66). New cstate: current_term: 4 leader_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:17.708200 10879 raft_consensus.cc:1275] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:17.709465 11175 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:17.718582 10954 raft_consensus.cc:1275] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:17.720014 10876 raft_consensus.cc:1275] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:17.720988 11175 consensus_queue.cc:1048] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:17.721421 11217 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:17.741375 10876 raft_consensus.cc:1275] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:17.744277 11235 consensus_queue.cc:1048] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86d8ebbcaff4e76af8e3acb67207878" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:17.744680 11026 raft_consensus.cc:1275] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 4 FOLLOWER]: Refusing update from remote peer fd3e554dff2d4ac1841f6d8c267b8578: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:17.746256 11219 consensus_queue.cc:1048] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e17dd92ad52846bd93d4c3237acd28e0" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 44365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:17.759423 10952 raft_consensus.cc:1275] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Refusing update from remote peer e17dd92ad52846bd93d4c3237acd28e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:17.760939 11225 consensus_queue.cc:1048] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3e554dff2d4ac1841f6d8c267b8578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:17.804037 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:17.809989 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
W20260709 13:14:17.817890 10919 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20260709 13:14:17.847126 10919 consensus_peers.cc:597] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 -> Peer d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757): Couldn't send request to peer d86d8ebbcaff4e76af8e3acb67207878. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:17.853017 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:17.853811 10257 tablet_replica.cc:333] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.854385 10257 raft_consensus.cc:2243] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.854876 10257 raft_consensus.cc:2272] T 0658ee47d6e4413e9f207532ea4999cc P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.856904 10257 tablet_replica.cc:333] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.857383 10257 raft_consensus.cc:2243] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:17.858012 10257 raft_consensus.cc:2272] T abd752c0c8fd42c0be6146dc0a7d1dbf P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.859614 10257 tablet_replica.cc:333] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.860087 10257 raft_consensus.cc:2243] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.860440 10257 raft_consensus.cc:2272] T a1bb61541b6b4b6f9d6be0312ba07c81 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.862013 10257 tablet_replica.cc:333] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.862445 10257 raft_consensus.cc:2243] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:17.863283 10257 raft_consensus.cc:2272] T 93e7d762c4d04bc0a93f865c0e4fffa9 P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.864938 10257 tablet_replica.cc:333] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.865433 10257 raft_consensus.cc:2243] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.865783 10257 raft_consensus.cc:2272] T 7946fded4dfd4022b386d9485b8ca5ce P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.868002 10257 tablet_replica.cc:333] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.868673 10257 raft_consensus.cc:2243] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:17.868759 10919 consensus_peers.cc:597] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 -> Peer d86d8ebbcaff4e76af8e3acb67207878 (127.10.4.65:44757): Couldn't send request to peer d86d8ebbcaff4e76af8e3acb67207878. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:17.869091 10257 raft_consensus.cc:2272] T 8634103b437b49a8b0d8724e44da474a P d86d8ebbcaff4e76af8e3acb67207878 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.870987 10257 tablet_replica.cc:333] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.871640 10257 raft_consensus.cc:2243] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:17.872378 10257 raft_consensus.cc:2272] T aada48f8534745c19e54a404fc4a3037 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.874078 10257 tablet_replica.cc:333] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.874524 10257 raft_consensus.cc:2243] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:14:17.875195 10257 raft_consensus.cc:2272] T 34941245c64f41ffb49311a76630551a P d86d8ebbcaff4e76af8e3acb67207878 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.876901 10257 tablet_replica.cc:333] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.877351 10257 raft_consensus.cc:2243] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.877789 10257 raft_consensus.cc:2272] T 3658791733f84b198f0385c5dac7cca2 P d86d8ebbcaff4e76af8e3acb67207878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.879632 10257 tablet_replica.cc:333] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878: stopping tablet replica
I20260709 13:14:17.880065 10257 raft_consensus.cc:2243] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.880457 10257 raft_consensus.cc:2272] T 42e64608e7c94b7caa8664d7aed15b50 P d86d8ebbcaff4e76af8e3acb67207878 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.901273 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
I20260709 13:14:17.918692 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
I20260709 13:14:17.940440 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:17.941208 10257 tablet_replica.cc:333] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.941998 10257 raft_consensus.cc:2243] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.942538 10257 raft_consensus.cc:2272] T 3658791733f84b198f0385c5dac7cca2 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.944307 10257 tablet_replica.cc:333] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.944797 10257 raft_consensus.cc:2243] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:17.945438 10257 raft_consensus.cc:2272] T 7946fded4dfd4022b386d9485b8ca5ce P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.947315 10257 tablet_replica.cc:333] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.947867 10257 raft_consensus.cc:2243] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:17.948645 10257 raft_consensus.cc:2272] T 8634103b437b49a8b0d8724e44da474a P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.950649 10257 tablet_replica.cc:333] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.951187 10257 raft_consensus.cc:2243] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.951709 10257 raft_consensus.cc:2272] T 93e7d762c4d04bc0a93f865c0e4fffa9 P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.953469 10257 tablet_replica.cc:333] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.953958 10257 raft_consensus.cc:2243] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.954360 10257 raft_consensus.cc:2272] T 42e64608e7c94b7caa8664d7aed15b50 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.956408 10257 tablet_replica.cc:333] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.956899 10257 raft_consensus.cc:2243] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.957300 10257 raft_consensus.cc:2272] T abd752c0c8fd42c0be6146dc0a7d1dbf P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.958770 10257 tablet_replica.cc:333] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.959200 10257 raft_consensus.cc:2243] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:17.960019 10257 raft_consensus.cc:2272] T a1bb61541b6b4b6f9d6be0312ba07c81 P fd3e554dff2d4ac1841f6d8c267b8578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.961512 10257 tablet_replica.cc:333] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.961948 10257 raft_consensus.cc:2243] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.962312 10257 raft_consensus.cc:2272] T 34941245c64f41ffb49311a76630551a P fd3e554dff2d4ac1841f6d8c267b8578 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.963819 10257 tablet_replica.cc:333] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.964217 10257 raft_consensus.cc:2243] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.964545 10257 raft_consensus.cc:2272] T aada48f8534745c19e54a404fc4a3037 P fd3e554dff2d4ac1841f6d8c267b8578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.966020 10257 tablet_replica.cc:333] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578: stopping tablet replica
I20260709 13:14:17.966403 10257 raft_consensus.cc:2243] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:17.966754 10257 raft_consensus.cc:2272] T 0658ee47d6e4413e9f207532ea4999cc P fd3e554dff2d4ac1841f6d8c267b8578 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:17.994078 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
I20260709 13:14:18.011089 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
I20260709 13:14:18.031242 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:18.031965 10257 tablet_replica.cc:333] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.032785 10257 raft_consensus.cc:2243] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:18.033710 10257 raft_consensus.cc:2272] T 3658791733f84b198f0385c5dac7cca2 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.035476 10257 tablet_replica.cc:333] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.035953 10257 raft_consensus.cc:2243] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:14:18.036634 10257 raft_consensus.cc:2272] T 0658ee47d6e4413e9f207532ea4999cc P e17dd92ad52846bd93d4c3237acd28e0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.038852 10257 tablet_replica.cc:333] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.039381 10257 raft_consensus.cc:2243] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:18.039863 10257 raft_consensus.cc:2272] T a1bb61541b6b4b6f9d6be0312ba07c81 P e17dd92ad52846bd93d4c3237acd28e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.041479 10257 tablet_replica.cc:333] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.041947 10257 raft_consensus.cc:2243] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:18.042341 10257 raft_consensus.cc:2272] T abd752c0c8fd42c0be6146dc0a7d1dbf P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.043915 10257 tablet_replica.cc:333] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.044384 10257 raft_consensus.cc:2243] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:18.044751 10257 raft_consensus.cc:2272] T 7946fded4dfd4022b386d9485b8ca5ce P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.046562 10257 tablet_replica.cc:333] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.047040 10257 raft_consensus.cc:2243] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:18.047529 10257 raft_consensus.cc:2272] T 8634103b437b49a8b0d8724e44da474a P e17dd92ad52846bd93d4c3237acd28e0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.049580 10257 tablet_replica.cc:333] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.050066 10257 raft_consensus.cc:2243] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:18.050506 10257 raft_consensus.cc:2272] T 93e7d762c4d04bc0a93f865c0e4fffa9 P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.052170 10257 tablet_replica.cc:333] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.052627 10257 raft_consensus.cc:2243] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:18.053002 10257 raft_consensus.cc:2272] T 34941245c64f41ffb49311a76630551a P e17dd92ad52846bd93d4c3237acd28e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.054778 10257 tablet_replica.cc:333] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.055258 10257 raft_consensus.cc:2243] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:18.055682 10257 raft_consensus.cc:2272] T aada48f8534745c19e54a404fc4a3037 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.057299 10257 tablet_replica.cc:333] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0: stopping tablet replica
I20260709 13:14:18.057698 10257 raft_consensus.cc:2243] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:14:18.058368 10257 raft_consensus.cc:2272] T 42e64608e7c94b7caa8664d7aed15b50 P e17dd92ad52846bd93d4c3237acd28e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.085678 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
I20260709 13:14:18.102944 10257 master.cc:562] Master@127.10.4.126:34809 shutting down...
I20260709 13:14:18.121603 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:14:18.122213 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:18.122573 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3e8a5b1ca7de4591a8761fc397773025: stopping tablet replica
I20260709 13:14:18.141963 10257 master.cc:584] Master@127.10.4.126:34809 shutdown complete.
d86d8ebbcaff4e76af8e3acb67207878  5
e17dd92ad52846bd93d4c3237acd28e0  1
fd3e554dff2d4ac1841f6d8c267b8578  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10571 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:14:18.178427 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:42409
I20260709 13:14:18.179584 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:18.184808 11243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:18.184794 11244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:18.187151 11246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:18.187868 10257 server_base.cc:1061] running on GCE node
I20260709 13:14:18.188798 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:18.188993 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:18.189164 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602858189145 us; error 0 us; skew 500 ppm
I20260709 13:14:18.189726 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:18.192562 10257 webserver.cc:533] Webserver started at http://127.10.4.126:36669/ using document root <none> and password file <none>
I20260709 13:14:18.193028 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:18.193197 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:18.193465 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:18.194639 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "ab9dad95b38845f197fb7c3c12a492c4"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.199138 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:14:18.202637 11251 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.203389 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:14:18.203719 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "ab9dad95b38845f197fb7c3c12a492c4"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.203994 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:18.243207 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:18.244508 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:18.281060 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:42409
I20260709 13:14:18.281149 11302 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:42409 every 8 connection(s)
I20260709 13:14:18.284718 11303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.294684 11303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Bootstrap starting.
I20260709 13:14:18.299566 11303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:18.304006 11303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: No bootstrap required, opened a new log
I20260709 13:14:18.306212 11303 raft_consensus.cc:359] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9dad95b38845f197fb7c3c12a492c4" member_type: VOTER }
I20260709 13:14:18.306591 11303 raft_consensus.cc:385] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:18.306766 11303 raft_consensus.cc:740] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab9dad95b38845f197fb7c3c12a492c4, State: Initialized, Role: FOLLOWER
I20260709 13:14:18.307258 11303 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9dad95b38845f197fb7c3c12a492c4" member_type: VOTER }
I20260709 13:14:18.307705 11303 raft_consensus.cc:399] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:14:18.307956 11303 raft_consensus.cc:493] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:14:18.308269 11303 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:18.312846 11303 raft_consensus.cc:515] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9dad95b38845f197fb7c3c12a492c4" member_type: VOTER }
I20260709 13:14:18.313421 11303 leader_election.cc:304] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab9dad95b38845f197fb7c3c12a492c4; no voters: 
I20260709 13:14:18.314640 11303 leader_election.cc:290] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:14:18.314996 11306 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:18.316354 11306 raft_consensus.cc:697] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 1 LEADER]: Becoming Leader. State: Replica: ab9dad95b38845f197fb7c3c12a492c4, State: Running, Role: LEADER
I20260709 13:14:18.317081 11306 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9dad95b38845f197fb7c3c12a492c4" member_type: VOTER }
I20260709 13:14:18.317692 11303 sys_catalog.cc:565] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:14:18.322220 11307 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab9dad95b38845f197fb7c3c12a492c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9dad95b38845f197fb7c3c12a492c4" member_type: VOTER } }
I20260709 13:14:18.322186 11308 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9dad95b38845f197fb7c3c12a492c4. Latest consensus state: current_term: 1 leader_uuid: "ab9dad95b38845f197fb7c3c12a492c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9dad95b38845f197fb7c3c12a492c4" member_type: VOTER } }
I20260709 13:14:18.323061 11307 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:18.323094 11308 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [sys.catalog]: This master's current role is: LEADER
I20260709 13:14:18.327587 11313 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:14:18.333127 11313 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:14:18.335726 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:14:18.341784 11313 catalog_manager.cc:1383] Generated new cluster ID: 8aea58970a244f2ea94664e2e0c2f3ec
I20260709 13:14:18.342075 11313 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:14:18.364974 11313 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:14:18.366159 11313 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:14:18.385123 11313 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Generated new TSK 0
I20260709 13:14:18.385720 11313 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:14:18.401906 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:18.407467 11324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:18.408617 11325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:18.410085 11327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:18.410399 10257 server_base.cc:1061] running on GCE node
I20260709 13:14:18.411203 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:18.411369 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:18.411557 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602858411540 us; error 0 us; skew 500 ppm
I20260709 13:14:18.412065 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:18.414111 10257 webserver.cc:533] Webserver started at http://127.10.4.65:43797/ using document root <none> and password file <none>
I20260709 13:14:18.414541 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:18.414712 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:18.414960 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:18.415983 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "01ae50c877d04ba687e4e06259ba5581"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.420230 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:14:18.423097 11332 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.423776 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:18.424026 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "01ae50c877d04ba687e4e06259ba5581"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.424295 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:18.434073 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:18.435190 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:18.436542 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:18.438568 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:18.438743 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.438956 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:18.439108 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.476166 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:42857
I20260709 13:14:18.476262 11394 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:42857 every 8 connection(s)
I20260709 13:14:18.480588 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:18.487437 11399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:18.488860 11400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:18.491268 11402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:18.492231 10257 server_base.cc:1061] running on GCE node
I20260709 13:14:18.493474 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:18.493645 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:18.493836 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602858493817 us; error 0 us; skew 500 ppm
I20260709 13:14:18.494417 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:18.494513 11395 heartbeater.cc:344] Connected to a master server at 127.10.4.126:42409
I20260709 13:14:18.494867 11395 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:18.495764 11395 heartbeater.cc:507] Master 127.10.4.126:42409 requested a full tablet report, sending...
I20260709 13:14:18.496763 10257 webserver.cc:533] Webserver started at http://127.10.4.66:33213/ using document root <none> and password file <none>
I20260709 13:14:18.497315 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:18.497537 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:18.497856 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:18.497879 11268 ts_manager.cc:194] Registered new tserver with Master: 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:18.499383 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "6e87b213a4554d94825aceec352fc094"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.500208 11268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39438
I20260709 13:14:18.504026 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:14:18.507267 11407 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.508229 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:14:18.508479 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "6e87b213a4554d94825aceec352fc094"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.508735 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:18.527786 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:18.529172 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:18.530619 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:18.532944 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:18.533161 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.533423 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:18.533564 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.570174 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:44257
I20260709 13:14:18.570262 11469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:44257 every 8 connection(s)
I20260709 13:14:18.574555 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:18.581009 11473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:18.582203 11474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:18.585561 10257 server_base.cc:1061] running on GCE node
W20260709 13:14:18.585690 11476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:18.587255 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:18.587535 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:18.587507 11470 heartbeater.cc:344] Connected to a master server at 127.10.4.126:42409
I20260709 13:14:18.587881 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602858587860 us; error 0 us; skew 500 ppm
I20260709 13:14:18.588043 11470 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:18.588472 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:18.588766 11470 heartbeater.cc:507] Master 127.10.4.126:42409 requested a full tablet report, sending...
I20260709 13:14:18.590763 11268 ts_manager.cc:194] Registered new tserver with Master: 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:18.591049 10257 webserver.cc:533] Webserver started at http://127.10.4.67:46017/ using document root <none> and password file <none>
I20260709 13:14:18.591602 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:18.591773 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:18.592031 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:18.592259 11268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39444
I20260709 13:14:18.593228 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "34a42aa6aa0a4778a532c6433d70b981"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.597869 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:14:18.600993 11481 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.601732 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:14:18.602010 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "34a42aa6aa0a4778a532c6433d70b981"
format_stamp: "Formatted at 2026-07-09 13:14:18 on dist-test-slave-gkm9"
I20260709 13:14:18.602273 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:18.615056 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:18.616247 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:18.617646 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:18.619843 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:18.620033 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.620256 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:18.620400 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:18.658381 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:46667
I20260709 13:14:18.658473 11543 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:46667 every 8 connection(s)
I20260709 13:14:18.671370 11544 heartbeater.cc:344] Connected to a master server at 127.10.4.126:42409
I20260709 13:14:18.671897 11544 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:18.672649 11544 heartbeater.cc:507] Master 127.10.4.126:42409 requested a full tablet report, sending...
I20260709 13:14:18.674461 11268 ts_manager.cc:194] Registered new tserver with Master: 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:18.674539 10257 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013203085s
I20260709 13:14:18.675806 11268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39460
I20260709 13:14:18.699144 11268 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39466:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:14:18.705633 11268 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:14:18.893728 11509 tablet_service.cc:1511] Processing CreateTablet for tablet bcf62f83a1754dcc80291daa521a9c06 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:14:18.895150 11509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf62f83a1754dcc80291daa521a9c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.897536 11508 tablet_service.cc:1511] Processing CreateTablet for tablet 44cbd38458af4b03b0d3f362bd6a2f85 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:14:18.898831 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44cbd38458af4b03b0d3f362bd6a2f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.899372 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 6b402b68ac7b4b9292a6ab6fb6fd3ea4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:14:18.900696 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b402b68ac7b4b9292a6ab6fb6fd3ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.902271 11505 tablet_service.cc:1511] Processing CreateTablet for tablet e5b7e70b18df44b6b0e248fbc3664d96 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:14:18.903657 11505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b7e70b18df44b6b0e248fbc3664d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.904701 11507 tablet_service.cc:1511] Processing CreateTablet for tablet a7effda9a8a1409cbe25d55ed7cf2964 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:14:18.906126 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7effda9a8a1409cbe25d55ed7cf2964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.908239 11435 tablet_service.cc:1511] Processing CreateTablet for tablet bcf62f83a1754dcc80291daa521a9c06 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:14:18.909791 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf62f83a1754dcc80291daa521a9c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.909891 11504 tablet_service.cc:1511] Processing CreateTablet for tablet df8439ad70bf422daa65806caf52f7f4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:14:18.923286 11504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df8439ad70bf422daa65806caf52f7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.924193 11503 tablet_service.cc:1511] Processing CreateTablet for tablet 38ecdb2d3e8745889c0343749502c9f5 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:14:18.925688 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ecdb2d3e8745889c0343749502c9f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.911468 11434 tablet_service.cc:1511] Processing CreateTablet for tablet 44cbd38458af4b03b0d3f362bd6a2f85 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:14:18.911769 11360 tablet_service.cc:1511] Processing CreateTablet for tablet bcf62f83a1754dcc80291daa521a9c06 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:14:18.928824 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf62f83a1754dcc80291daa521a9c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.932586 11355 tablet_service.cc:1511] Processing CreateTablet for tablet df8439ad70bf422daa65806caf52f7f4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:14:18.933859 11355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df8439ad70bf422daa65806caf52f7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.936559 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 38ecdb2d3e8745889c0343749502c9f5 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:14:18.936861 11429 tablet_service.cc:1511] Processing CreateTablet for tablet 38ecdb2d3e8745889c0343749502c9f5 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:14:18.937821 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ecdb2d3e8745889c0343749502c9f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.913355 11433 tablet_service.cc:1511] Processing CreateTablet for tablet a7effda9a8a1409cbe25d55ed7cf2964 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:14:18.940408 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 3d475300e0054608a5d243d859d6609b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:14:18.941694 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d475300e0054608a5d243d859d6609b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.941565 11428 tablet_service.cc:1511] Processing CreateTablet for tablet 3d475300e0054608a5d243d859d6609b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:14:18.942751 11428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d475300e0054608a5d243d859d6609b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.915375 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 6b402b68ac7b4b9292a6ab6fb6fd3ea4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:14:18.943786 11429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ecdb2d3e8745889c0343749502c9f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.915839 11358 tablet_service.cc:1511] Processing CreateTablet for tablet a7effda9a8a1409cbe25d55ed7cf2964 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:14:18.944734 11358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7effda9a8a1409cbe25d55ed7cf2964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.917209 11431 tablet_service.cc:1511] Processing CreateTablet for tablet e5b7e70b18df44b6b0e248fbc3664d96 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:14:18.947074 11431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b7e70b18df44b6b0e248fbc3664d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.947806 11433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7effda9a8a1409cbe25d55ed7cf2964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.920275 11357 tablet_service.cc:1511] Processing CreateTablet for tablet 6b402b68ac7b4b9292a6ab6fb6fd3ea4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:14:18.948873 11357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b402b68ac7b4b9292a6ab6fb6fd3ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.921808 11356 tablet_service.cc:1511] Processing CreateTablet for tablet e5b7e70b18df44b6b0e248fbc3664d96 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:14:18.951804 11434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44cbd38458af4b03b0d3f362bd6a2f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.952276 11356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b7e70b18df44b6b0e248fbc3664d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.916718 11359 tablet_service.cc:1511] Processing CreateTablet for tablet 44cbd38458af4b03b0d3f362bd6a2f85 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:14:18.954383 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 3d475300e0054608a5d243d859d6609b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:14:18.954931 11359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44cbd38458af4b03b0d3f362bd6a2f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.955695 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d475300e0054608a5d243d859d6609b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.960582 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:14:18.961869 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.918768 11430 tablet_service.cc:1511] Processing CreateTablet for tablet df8439ad70bf422daa65806caf52f7f4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:14:18.964252 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df8439ad70bf422daa65806caf52f7f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.969901 11427 tablet_service.cc:1511] Processing CreateTablet for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:14:18.971272 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.971916 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b402b68ac7b4b9292a6ab6fb6fd3ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.986608 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:14:18.988022 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.988797 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 98fc3c2677604f3bb787b67c5e5b6012 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:14:18.990190 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98fc3c2677604f3bb787b67c5e5b6012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:18.991089 11426 tablet_service.cc:1511] Processing CreateTablet for tablet 98fc3c2677604f3bb787b67c5e5b6012 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:14:18.992488 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98fc3c2677604f3bb787b67c5e5b6012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.006067 11503 tablet_service.cc:1511] Processing CreateTablet for tablet 98fc3c2677604f3bb787b67c5e5b6012 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:14:19.007825 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98fc3c2677604f3bb787b67c5e5b6012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.011914 11560 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:19.031690 11435 tablet_service.cc:1511] Processing CreateTablet for tablet 593d6a54177d461680000f43998f8775 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:14:19.033083 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593d6a54177d461680000f43998f8775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.047447 11508 tablet_service.cc:1511] Processing CreateTablet for tablet 593d6a54177d461680000f43998f8775 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:14:19.049464 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593d6a54177d461680000f43998f8775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.054767 11427 tablet_service.cc:1511] Processing CreateTablet for tablet 2eaec0f849964172b0dcfba0af0e7fa3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:14:19.056252 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eaec0f849964172b0dcfba0af0e7fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.077225 11560 tablet_bootstrap.cc:654] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.079783 11508 tablet_service.cc:1511] Processing CreateTablet for tablet 2eaec0f849964172b0dcfba0af0e7fa3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:14:19.079775 11559 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:19.082453 11358 tablet_service.cc:1511] Processing CreateTablet for tablet 593d6a54177d461680000f43998f8775 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:14:19.088476 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 2eaec0f849964172b0dcfba0af0e7fa3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:14:19.091377 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eaec0f849964172b0dcfba0af0e7fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.092741 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 3cd0c27e51bd41b5959c50a56353166e (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:14:19.088232 11507 tablet_service.cc:1511] Processing CreateTablet for tablet 3cd0c27e51bd41b5959c50a56353166e (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:14:19.096786 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd0c27e51bd41b5959c50a56353166e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.102550 11358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593d6a54177d461680000f43998f8775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.103595 11561 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:19.107964 11503 tablet_service.cc:1511] Processing CreateTablet for tablet 43fc774e6f3a42cf890bd8e9fb8f7635 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:14:19.106894 11435 tablet_service.cc:1511] Processing CreateTablet for tablet 43fc774e6f3a42cf890bd8e9fb8f7635 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:14:19.109735 11430 tablet_service.cc:1511] Processing CreateTablet for tablet 3cd0c27e51bd41b5959c50a56353166e (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:14:19.114841 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fc774e6f3a42cf890bd8e9fb8f7635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.115338 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2eaec0f849964172b0dcfba0af0e7fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.115764 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fc774e6f3a42cf890bd8e9fb8f7635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.117481 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd0c27e51bd41b5959c50a56353166e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.119549 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd0c27e51bd41b5959c50a56353166e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.108549 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 43fc774e6f3a42cf890bd8e9fb8f7635 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:14:19.122781 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fc774e6f3a42cf890bd8e9fb8f7635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.127893 11431 tablet_service.cc:1511] Processing CreateTablet for tablet 81760accce3a440a82323bd3ff53c551 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:14:19.131705 11431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81760accce3a440a82323bd3ff53c551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.129436 11559 tablet_bootstrap.cc:654] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.131825 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 81760accce3a440a82323bd3ff53c551 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:14:19.133100 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81760accce3a440a82323bd3ff53c551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.110903 11357 tablet_service.cc:1511] Processing CreateTablet for tablet 81760accce3a440a82323bd3ff53c551 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:14:19.138554 11354 tablet_service.cc:1511] Processing CreateTablet for tablet e4aa2e377a0f45378e0df4a788c07ddf (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:14:19.144989 11426 tablet_service.cc:1511] Processing CreateTablet for tablet e4aa2e377a0f45378e0df4a788c07ddf (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:14:19.146222 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4aa2e377a0f45378e0df4a788c07ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.147719 11505 tablet_service.cc:1511] Processing CreateTablet for tablet e4aa2e377a0f45378e0df4a788c07ddf (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:14:19.148972 11505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4aa2e377a0f45378e0df4a788c07ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.149889 11561 tablet_bootstrap.cc:654] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.147691 11357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81760accce3a440a82323bd3ff53c551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.155726 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4aa2e377a0f45378e0df4a788c07ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.156464 11353 tablet_service.cc:1511] Processing CreateTablet for tablet dbbdba17cce24fe7923cf755be2d3e0a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:14:19.157711 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbbdba17cce24fe7923cf755be2d3e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.163970 11432 tablet_service.cc:1511] Processing CreateTablet for tablet dbbdba17cce24fe7923cf755be2d3e0a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:14:19.165249 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbbdba17cce24fe7923cf755be2d3e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.176409 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 1073b5e444bf4b34ad3dca3ea96efc76 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:14:19.177985 11508 tablet_service.cc:1511] Processing CreateTablet for tablet dbbdba17cce24fe7923cf755be2d3e0a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:14:19.180373 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbbdba17cce24fe7923cf755be2d3e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.178797 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1073b5e444bf4b34ad3dca3ea96efc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.185185 11560 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:19.185662 11560 ts_tablet_manager.cc:1403] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.174s	user 0.014s	sys 0.015s
I20260709 13:14:19.189481 11559 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:19.189234 11560 raft_consensus.cc:359] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.190001 11559 ts_tablet_manager.cc:1403] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.111s	user 0.011s	sys 0.004s
I20260709 13:14:19.192559 11560 raft_consensus.cc:385] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.195783 11560 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.196623 11560 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.198489 11559 raft_consensus.cc:359] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.204288 11559 raft_consensus.cc:385] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.204638 11559 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.205633 11559 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.213794 11507 tablet_service.cc:1511] Processing CreateTablet for tablet 1073b5e444bf4b34ad3dca3ea96efc76 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:14:19.214293 11427 tablet_service.cc:1511] Processing CreateTablet for tablet 988262f8a4a846be8b350d9d99c4a730 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:14:19.215456 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 988262f8a4a846be8b350d9d99c4a730 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:14:19.218182 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988262f8a4a846be8b350d9d99c4a730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.218935 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988262f8a4a846be8b350d9d99c4a730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.218182 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1073b5e444bf4b34ad3dca3ea96efc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.219914 11470 heartbeater.cc:499] Master 127.10.4.126:42409 was elected leader, sending a full tablet report...
I20260709 13:14:19.216900 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 1073b5e444bf4b34ad3dca3ea96efc76 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:14:19.226665 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1073b5e444bf4b34ad3dca3ea96efc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.240506 11561 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:19.240936 11544 heartbeater.cc:499] Master 127.10.4.126:42409 was elected leader, sending a full tablet report...
I20260709 13:14:19.236650 11560 ts_tablet_manager.cc:1434] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.051s	user 0.012s	sys 0.000s
I20260709 13:14:19.244325 11560 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:19.247496 11561 ts_tablet_manager.cc:1403] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.146s	user 0.017s	sys 0.024s
I20260709 13:14:19.242341 11559 ts_tablet_manager.cc:1434] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.002s
I20260709 13:14:19.251633 11559 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:19.252794 11566 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.253333 11503 tablet_service.cc:1511] Processing CreateTablet for tablet 988262f8a4a846be8b350d9d99c4a730 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:14:19.254597 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988262f8a4a846be8b350d9d99c4a730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.253335 11566 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.258196 11559 tablet_bootstrap.cc:654] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.263995 11561 raft_consensus.cc:359] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.266882 11561 raft_consensus.cc:385] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.268918 11561 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.270277 11560 tablet_bootstrap.cc:654] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.269903 11561 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.274669 11566 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:19.277182 11357 tablet_service.cc:1511] Processing CreateTablet for tablet ab2785abe6bd4898ac2dbb25f53e4c8b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:14:19.282903 11357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab2785abe6bd4898ac2dbb25f53e4c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.285053 11426 tablet_service.cc:1511] Processing CreateTablet for tablet ab2785abe6bd4898ac2dbb25f53e4c8b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:14:19.293599 11508 tablet_service.cc:1511] Processing CreateTablet for tablet ab2785abe6bd4898ac2dbb25f53e4c8b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:14:19.294941 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab2785abe6bd4898ac2dbb25f53e4c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.295547 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab2785abe6bd4898ac2dbb25f53e4c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.306694 11561 ts_tablet_manager.cc:1434] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.006s
I20260709 13:14:19.307555 11395 heartbeater.cc:499] Master 127.10.4.126:42409 was elected leader, sending a full tablet report...
I20260709 13:14:19.309706 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:14:19.312704 11484 leader_election.cc:343] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.313515 11559 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:19.313987 11559 ts_tablet_manager.cc:1403] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.063s	user 0.003s	sys 0.008s
I20260709 13:14:19.315001 11561 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:19.316849 11559 raft_consensus.cc:359] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.317557 11559 raft_consensus.cc:385] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.317857 11559 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.314960 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 741facf7b8264a10bc5c3431415fc34a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:14:19.320761 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741facf7b8264a10bc5c3431415fc34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.322165 11561 tablet_bootstrap.cc:654] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.329885 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 741facf7b8264a10bc5c3431415fc34a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:14:19.331215 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741facf7b8264a10bc5c3431415fc34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.330915 11559 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.335005 11430 tablet_service.cc:1511] Processing CreateTablet for tablet ae095c448d6e4a3ab9eaecd9695887a7 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:14:19.336297 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae095c448d6e4a3ab9eaecd9695887a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.337464 11559 ts_tablet_manager.cc:1434] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:14:19.332499 11507 tablet_service.cc:1511] Processing CreateTablet for tablet 741facf7b8264a10bc5c3431415fc34a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
W20260709 13:14:19.332002 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:19.340741 11352 tablet_service.cc:1511] Processing CreateTablet for tablet ae095c448d6e4a3ab9eaecd9695887a7 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:14:19.342212 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae095c448d6e4a3ab9eaecd9695887a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.346241 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:19.346429 11503 tablet_service.cc:1511] Processing CreateTablet for tablet ae095c448d6e4a3ab9eaecd9695887a7 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
W20260709 13:14:19.352486 11483 leader_election.cc:343] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.354034 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:19.355268 11566 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:19.359993 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741facf7b8264a10bc5c3431415fc34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.363433 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 435f2e223e784d188897ac4adbc34211 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:14:19.364281 11559 tablet_bootstrap.cc:492] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:19.364972 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae095c448d6e4a3ab9eaecd9695887a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.364982 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435f2e223e784d188897ac4adbc34211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.373903 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 435f2e223e784d188897ac4adbc34211 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:14:19.374147 11427 tablet_service.cc:1511] Processing CreateTablet for tablet 435f2e223e784d188897ac4adbc34211 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:14:19.375226 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435f2e223e784d188897ac4adbc34211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.376471 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435f2e223e784d188897ac4adbc34211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.387280 11358 tablet_service.cc:1511] Processing CreateTablet for tablet 81ba42e5875044cb8ebec6e017d5585b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:14:19.389005 11358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ba42e5875044cb8ebec6e017d5585b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.390132 11508 tablet_service.cc:1511] Processing CreateTablet for tablet 81ba42e5875044cb8ebec6e017d5585b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:14:19.395843 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ba42e5875044cb8ebec6e017d5585b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.395843 11430 tablet_service.cc:1511] Processing CreateTablet for tablet 81ba42e5875044cb8ebec6e017d5585b (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:14:19.396694 11560 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:19.398491 11560 ts_tablet_manager.cc:1403] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.154s	user 0.012s	sys 0.007s
I20260709 13:14:19.398779 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 114b7c3bbf65457b98b17b61f3c64839 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:14:19.400058 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114b7c3bbf65457b98b17b61f3c64839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.401341 11560 raft_consensus.cc:359] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.402040 11560 raft_consensus.cc:385] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.402326 11560 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.408365 11561 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:19.408893 11561 ts_tablet_manager.cc:1403] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.094s	user 0.032s	sys 0.030s
I20260709 13:14:19.415763 11505 tablet_service.cc:1511] Processing CreateTablet for tablet 114b7c3bbf65457b98b17b61f3c64839 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:14:19.417074 11505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114b7c3bbf65457b98b17b61f3c64839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.420135 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 114b7c3bbf65457b98b17b61f3c64839 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:14:19.422212 11559 tablet_bootstrap.cc:654] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.415681 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ba42e5875044cb8ebec6e017d5585b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.404915 11560 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.427650 11560 ts_tablet_manager.cc:1434] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.002s
I20260709 13:14:19.428553 11560 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:19.428367 11354 tablet_service.cc:1511] Processing CreateTablet for tablet ad3ab48ae4bf4452a19c99c71fd014f1 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:14:19.429554 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad3ab48ae4bf4452a19c99c71fd014f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.434974 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114b7c3bbf65457b98b17b61f3c64839. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.437791 11427 tablet_service.cc:1511] Processing CreateTablet for tablet ad3ab48ae4bf4452a19c99c71fd014f1 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:14:19.438997 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad3ab48ae4bf4452a19c99c71fd014f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.431879 11561 raft_consensus.cc:359] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.448050 11561 raft_consensus.cc:385] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.448417 11561 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.449326 11561 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.451781 11561 ts_tablet_manager.cc:1434] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:14:19.452800 11561 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:19.461977 11565 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.467674 11565 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.470305 11565 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:19.465534 11559 tablet_bootstrap.cc:492] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:19.471159 11559 ts_tablet_manager.cc:1403] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.107s	user 0.009s	sys 0.003s
I20260709 13:14:19.465283 11426 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd69c97ea4640a0a71117011ab4e5d4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:14:19.472245 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd69c97ea4640a0a71117011ab4e5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.474447 11560 tablet_bootstrap.cc:654] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.466869 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd69c97ea4640a0a71117011ab4e5d4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:14:19.476246 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd69c97ea4640a0a71117011ab4e5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.478111 11559 raft_consensus.cc:359] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.478987 11559 raft_consensus.cc:385] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.479322 11559 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.480214 11559 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.495697 11570 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.496222 11570 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.507041 11570 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:19.494792 11559 ts_tablet_manager.cc:1434] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:14:19.492146 11503 tablet_service.cc:1511] Processing CreateTablet for tablet ad3ab48ae4bf4452a19c99c71fd014f1 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:14:19.558753 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:19.560577 11409 leader_election.cc:343] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.562585 11508 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd69c97ea4640a0a71117011ab4e5d4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:14:19.564078 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd69c97ea4640a0a71117011ab4e5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.566970 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 063871012da74857a239cdf01c76e5ab (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:14:19.568593 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063871012da74857a239cdf01c76e5ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.568408 11430 tablet_service.cc:1511] Processing CreateTablet for tablet 063871012da74857a239cdf01c76e5ab (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:14:19.569617 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063871012da74857a239cdf01c76e5ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.570840 11559 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:19.571485 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:14:19.572681 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.576725 11505 tablet_service.cc:1511] Processing CreateTablet for tablet 063871012da74857a239cdf01c76e5ab (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:14:19.578100 11505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063871012da74857a239cdf01c76e5ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.582629 11561 tablet_bootstrap.cc:654] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.584631 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:19.586385 11408 leader_election.cc:343] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.586818 11408 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:19.588021 11565 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:19.591184 11427 tablet_service.cc:1511] Processing CreateTablet for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:14:19.592504 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.592319 11507 tablet_service.cc:1511] Processing CreateTablet for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:14:19.593622 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.600431 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad3ab48ae4bf4452a19c99c71fd014f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.602525 11354 tablet_service.cc:1511] Processing CreateTablet for tablet bce024e6afd54cf0992c0af40df6a4eb (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:14:19.603758 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce024e6afd54cf0992c0af40df6a4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.608314 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:14:19.608825 11566 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.609676 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.609315 11566 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.612581 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:19.612859 11426 tablet_service.cc:1511] Processing CreateTablet for tablet bce024e6afd54cf0992c0af40df6a4eb (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:14:19.613593 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:14:19.614091 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce024e6afd54cf0992c0af40df6a4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.614370 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:19.614825 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.615003 11445 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:19.607081 11506 tablet_service.cc:1511] Processing CreateTablet for tablet bce024e6afd54cf0992c0af40df6a4eb (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:14:19.616384 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce024e6afd54cf0992c0af40df6a4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.609251 11504 tablet_service.cc:1511] Processing CreateTablet for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:14:19.616451 11484 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:19.617388 11370 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:19.617723 11358 tablet_service.cc:1511] Processing CreateTablet for tablet 116f1226682347599c239ad3bee95b41 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:14:19.611384 11566 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:19.618902 11358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116f1226682347599c239ad3bee95b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.624460 11566 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.624974 11566 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.625381 11566 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.626547 11431 tablet_service.cc:1511] Processing CreateTablet for tablet 116f1226682347599c239ad3bee95b41 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:14:19.627907 11431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116f1226682347599c239ad3bee95b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.630465 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 8b949bd9634b40af97ef4fb06ee21e9f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:14:19.631700 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b949bd9634b40af97ef4fb06ee21e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.631346 11509 tablet_service.cc:1511] Processing CreateTablet for tablet 116f1226682347599c239ad3bee95b41 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:14:19.632663 11509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116f1226682347599c239ad3bee95b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.643117 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:14:19.644913 11335 leader_election.cc:343] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.657184 11504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.657989 11435 tablet_service.cc:1511] Processing CreateTablet for tablet 8b949bd9634b40af97ef4fb06ee21e9f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:14:19.659371 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b949bd9634b40af97ef4fb06ee21e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.664462 11560 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:19.664917 11560 ts_tablet_manager.cc:1403] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.237s	user 0.031s	sys 0.074s
I20260709 13:14:19.667115 11560 raft_consensus.cc:359] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.667695 11560 raft_consensus.cc:385] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.667981 11560 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.668829 11560 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.673436 11560 ts_tablet_manager.cc:1434] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:14:19.674377 11560 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:19.674906 11559 tablet_bootstrap.cc:654] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.684943 11509 tablet_service.cc:1511] Processing CreateTablet for tablet 8b949bd9634b40af97ef4fb06ee21e9f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:14:19.694316 11509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b949bd9634b40af97ef4fb06ee21e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.693989 11357 tablet_service.cc:1511] Processing CreateTablet for tablet b01b3301ee794d99bcb4feb672ac699f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:14:19.695268 11357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b01b3301ee794d99bcb4feb672ac699f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.696506 11560 tablet_bootstrap.cc:654] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.698604 11507 tablet_service.cc:1511] Processing CreateTablet for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:14:19.693073 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:14:19.701931 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.703639 11561 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:19.704306 11561 ts_tablet_manager.cc:1403] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.252s	user 0.013s	sys 0.002s
I20260709 13:14:19.707573 11561 raft_consensus.cc:359] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.708305 11561 raft_consensus.cc:385] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.708647 11561 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.712941 11430 tablet_service.cc:1511] Processing CreateTablet for tablet b01b3301ee794d99bcb4feb672ac699f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:14:19.713828 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 7d370e878a484e1aaaa0df1466da6f24 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:14:19.714988 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d370e878a484e1aaaa0df1466da6f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.716003 11433 tablet_service.cc:1511] Processing CreateTablet for tablet 7d370e878a484e1aaaa0df1466da6f24 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:14:19.714358 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b01b3301ee794d99bcb4feb672ac699f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.732653 11433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d370e878a484e1aaaa0df1466da6f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.736004 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.736694 11561 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.715106 11566 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.751130 11566 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:19.752008 11431 tablet_service.cc:1511] Processing CreateTablet for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:14:19.752293 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:19.752832 11370 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.752894 11561 ts_tablet_manager.cc:1434] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.048s	user 0.002s	sys 0.004s
I20260709 13:14:19.753309 11431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.753916 11561 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:19.759233 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:19.759836 11445 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.766703 11560 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:19.766361 11502 tablet_service.cc:1511] Processing CreateTablet for tablet 7d370e878a484e1aaaa0df1466da6f24 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:14:19.767225 11560 ts_tablet_manager.cc:1403] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.093s	user 0.005s	sys 0.009s
I20260709 13:14:19.767647 11561 tablet_bootstrap.cc:654] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.767930 11502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d370e878a484e1aaaa0df1466da6f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.770074 11560 raft_consensus.cc:359] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.770738 11560 raft_consensus.cc:385] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.771003 11560 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.771778 11503 tablet_service.cc:1511] Processing CreateTablet for tablet b01b3301ee794d99bcb4feb672ac699f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:14:19.772948 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b01b3301ee794d99bcb4feb672ac699f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.773988 11370 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:19.775746 11560 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.777535 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:19.778473 11566 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:19.780112 11560 ts_tablet_manager.cc:1434] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:14:19.780961 11445 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:19.797001 11560 tablet_bootstrap.cc:492] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:19.802340 11426 tablet_service.cc:1511] Processing CreateTablet for tablet 887116b7f3e447028e317f03386bf8d0 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:14:19.804396 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3368fb0be64c3b92da009af48a1854 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:14:19.804921 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887116b7f3e447028e317f03386bf8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.805652 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3368fb0be64c3b92da009af48a1854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.803104 11560 tablet_bootstrap.cc:654] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.813414 11559 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:19.814035 11559 ts_tablet_manager.cc:1403] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.243s	user 0.026s	sys 0.015s
I20260709 13:14:19.813880 11427 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3368fb0be64c3b92da009af48a1854 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:14:19.815124 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3368fb0be64c3b92da009af48a1854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.819765 11504 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3368fb0be64c3b92da009af48a1854 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:14:19.821030 11504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3368fb0be64c3b92da009af48a1854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.821491 11559 raft_consensus.cc:359] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.822328 11559 raft_consensus.cc:385] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:19.822681 11559 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:19.823537 11559 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.823994 11508 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5787304dfe4697baeaeb367ee7dda3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:14:19.830655 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5787304dfe4697baeaeb367ee7dda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.830569 11505 tablet_service.cc:1511] Processing CreateTablet for tablet 857937fa61dc400a81345944f0a14511 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:14:19.826851 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5787304dfe4697baeaeb367ee7dda3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:14:19.829337 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:19.832373 11519 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:19.829476 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 857937fa61dc400a81345944f0a14511 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:14:19.831934 11505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 857937fa61dc400a81345944f0a14511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.833663 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:14:19.832729 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5787304dfe4697baeaeb367ee7dda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.834959 11570 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.835462 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 857937fa61dc400a81345944f0a14511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.835366 11570 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.835938 11570 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.835027 11431 tablet_service.cc:1511] Processing CreateTablet for tablet 857937fa61dc400a81345944f0a14511 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:14:19.838236 11431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 857937fa61dc400a81345944f0a14511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.838124 11433 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5787304dfe4697baeaeb367ee7dda3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:14:19.840965 11433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5787304dfe4697baeaeb367ee7dda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.852514 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 887116b7f3e447028e317f03386bf8d0 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:14:19.854089 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887116b7f3e447028e317f03386bf8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.832521 11358 tablet_service.cc:1511] Processing CreateTablet for tablet 887116b7f3e447028e317f03386bf8d0 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:14:19.860890 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 7074ddeb0b7b4c1395b798352ff7db0a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:14:19.861763 11358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887116b7f3e447028e317f03386bf8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.862418 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7074ddeb0b7b4c1395b798352ff7db0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.872661 11435 tablet_service.cc:1511] Processing CreateTablet for tablet 7074ddeb0b7b4c1395b798352ff7db0a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:14:19.873970 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7074ddeb0b7b4c1395b798352ff7db0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.891427 11570 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.896948 11570 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:19.920841 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:19.921116 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 683aea8d9bd64288b5932ad265b60dd4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:14:19.921446 11519 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.899781 11566 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:19.928988 11508 tablet_service.cc:1511] Processing CreateTablet for tablet 683aea8d9bd64288b5932ad265b60dd4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:14:19.929996 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 683aea8d9bd64288b5932ad265b60dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.934764 11357 tablet_service.cc:1511] Processing CreateTablet for tablet 47922bb832834e72bc79e1e8159e6c1f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:14:19.935098 11502 tablet_service.cc:1511] Processing CreateTablet for tablet 47922bb832834e72bc79e1e8159e6c1f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:14:19.935981 11357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47922bb832834e72bc79e1e8159e6c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.936107 11502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47922bb832834e72bc79e1e8159e6c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.936214 11434 tablet_service.cc:1511] Processing CreateTablet for tablet 47922bb832834e72bc79e1e8159e6c1f (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:14:19.937669 11585 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.938107 11585 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.926618 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:19.927289 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 683aea8d9bd64288b5932ad265b60dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.939581 11434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47922bb832834e72bc79e1e8159e6c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.940088 11585 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:14:19.941352 11335 leader_election.cc:343] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.934134 11559 ts_tablet_manager.cc:1434] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.120s	user 0.016s	sys 0.016s
I20260709 13:14:19.942118 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:19.935158 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 683aea8d9bd64288b5932ad265b60dd4 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 13:14:19.943238 11484 leader_election.cc:343] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.943574 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 683aea8d9bd64288b5932ad265b60dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.944157 11559 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:19.948735 11354 tablet_service.cc:1511] Processing CreateTablet for tablet c753736d0d7f4578b8935b855a6b36e3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:14:19.951587 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c753736d0d7f4578b8935b855a6b36e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.956007 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:19.956849 11429 tablet_service.cc:1511] Processing CreateTablet for tablet c753736d0d7f4578b8935b855a6b36e3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:14:19.958040 11429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c753736d0d7f4578b8935b855a6b36e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.958381 11559 tablet_bootstrap.cc:654] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:19.927305 11509 tablet_service.cc:1511] Processing CreateTablet for tablet 7074ddeb0b7b4c1395b798352ff7db0a (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:14:19.959828 11483 leader_election.cc:343] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.964084 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:19.965047 11588 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:19.959923 11509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7074ddeb0b7b4c1395b798352ff7db0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.973074 11508 tablet_service.cc:1511] Processing CreateTablet for tablet c753736d0d7f4578b8935b855a6b36e3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:14:19.974839 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c753736d0d7f4578b8935b855a6b36e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:19.978576 11570 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.979032 11570 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.981033 11570 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:19.982486 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:19.983090 11445 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:19.984593 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:19.986078 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:19.987375 11333 leader_election.cc:343] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:19.987561 11566 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } }
I20260709 13:14:19.988197 11570 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:19.988580 11570 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.988891 11570 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:19.991071 11519 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:19.992251 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:14:19.995692 11565 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.996125 11565 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:19.998006 11565 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:19.999595 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:20.001716 11409 leader_election.cc:343] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.002100 11560 tablet_bootstrap.cc:492] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.002586 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:20.003298 11519 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:20.004448 11408 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:20.005515 11565 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:20.005862 11565 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.006184 11565 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.012331 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 7dab116931a64024ac61a35a6d29a836 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:14:20.013599 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dab116931a64024ac61a35a6d29a836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.002665 11561 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:20.018785 11561 ts_tablet_manager.cc:1403] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.265s	user 0.037s	sys 0.099s
I20260709 13:14:20.019332 11507 tablet_service.cc:1511] Processing CreateTablet for tablet 7dab116931a64024ac61a35a6d29a836 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:14:20.003258 11570 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.020623 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dab116931a64024ac61a35a6d29a836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.002566 11560 ts_tablet_manager.cc:1403] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.206s	user 0.015s	sys 0.009s
I20260709 13:14:20.035387 11590 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:14:20.036566 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 44376.8us
I20260709 13:14:20.037338 11565 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.040205 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:14:20.041528 11409 leader_election.cc:343] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.045974 11431 tablet_service.cc:1511] Processing CreateTablet for tablet 7dab116931a64024ac61a35a6d29a836 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:14:20.047329 11431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dab116931a64024ac61a35a6d29a836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.058419 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:20.059087 11519 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.068256 11565 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.068882 11565 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.069320 11565 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.041577 11560 raft_consensus.cc:359] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.040994 11561 raft_consensus.cc:359] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.082818 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 9c1dd88ce9a04ff59ee55a115686f148 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:14:20.085456 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1dd88ce9a04ff59ee55a115686f148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.088948 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:20.089488 11445 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.091084 11265 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.095650 11559 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:20.096093 11559 ts_tablet_manager.cc:1403] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.152s	user 0.008s	sys 0.012s
I20260709 13:14:20.084997 11560 raft_consensus.cc:385] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.099550 11560 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.100214 11570 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.102548 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:20.098814 11559 raft_consensus.cc:359] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.087693 11561 raft_consensus.cc:385] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.110857 11426 tablet_service.cc:1511] Processing CreateTablet for tablet 9c1dd88ce9a04ff59ee55a115686f148 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:14:20.112112 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1dd88ce9a04ff59ee55a115686f148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.111644 11354 tablet_service.cc:1511] Processing CreateTablet for tablet bab171deb5e946c8b074a4c091542a78 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:14:20.114910 11357 tablet_service.cc:1511] Processing CreateTablet for tablet 99667fcf16014dc1b9b174f0a37d16af (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:14:20.117508 11427 tablet_service.cc:1511] Processing CreateTablet for tablet bab171deb5e946c8b074a4c091542a78 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:14:20.118125 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 99667fcf16014dc1b9b174f0a37d16af (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:14:20.118690 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab171deb5e946c8b074a4c091542a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.119232 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99667fcf16014dc1b9b174f0a37d16af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.120221 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 9c1dd88ce9a04ff59ee55a115686f148 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:14:20.120376 11508 tablet_service.cc:1511] Processing CreateTablet for tablet bab171deb5e946c8b074a4c091542a78 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:14:20.121447 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1dd88ce9a04ff59ee55a115686f148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.124367 11351 tablet_service.cc:1511] Processing CreateTablet for tablet 572d79fba792428db7fcc1994fff81f7 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:14:20.099767 11445 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:20.126694 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab171deb5e946c8b074a4c091542a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.126343 11430 tablet_service.cc:1511] Processing CreateTablet for tablet 1ea18eac305644ed8d44618c2c0a8d0d (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:14:20.126364 11435 tablet_service.cc:1511] Processing CreateTablet for tablet 572d79fba792428db7fcc1994fff81f7 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:14:20.127651 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:20.119112 11505 tablet_service.cc:1511] Processing CreateTablet for tablet 99667fcf16014dc1b9b174f0a37d16af (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:14:20.128326 11434 tablet_service.cc:1511] Processing CreateTablet for tablet cf199b70fbb94d728a09194d18ca3b1e (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:14:20.128989 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab171deb5e946c8b074a4c091542a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.129359 11505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99667fcf16014dc1b9b174f0a37d16af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.129431 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ea18eac305644ed8d44618c2c0a8d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.103717 11590 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:20.106566 11559 raft_consensus.cc:385] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.137825 11360 tablet_service.cc:1511] Processing CreateTablet for tablet 1ea18eac305644ed8d44618c2c0a8d0d (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:14:20.139184 11356 tablet_service.cc:1511] Processing CreateTablet for tablet cf199b70fbb94d728a09194d18ca3b1e (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:14:20.139858 11351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d79fba792428db7fcc1994fff81f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.141049 11434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf199b70fbb94d728a09194d18ca3b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.151307 11356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf199b70fbb94d728a09194d18ca3b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.152680 11359 tablet_service.cc:1511] Processing CreateTablet for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:14:20.153923 11359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.148137 11559 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.141433 11503 tablet_service.cc:1511] Processing CreateTablet for tablet 572d79fba792428db7fcc1994fff81f7 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:14:20.154997 11559 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.156194 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d79fba792428db7fcc1994fff81f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.157501 11559 ts_tablet_manager.cc:1434] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.001s
I20260709 13:14:20.158505 11559 tablet_bootstrap.cc:492] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:20.142433 11504 tablet_service.cc:1511] Processing CreateTablet for tablet cf199b70fbb94d728a09194d18ca3b1e (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:14:20.160231 11504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf199b70fbb94d728a09194d18ca3b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.142813 11502 tablet_service.cc:1511] Processing CreateTablet for tablet 1ea18eac305644ed8d44618c2c0a8d0d (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:14:20.164170 11502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ea18eac305644ed8d44618c2c0a8d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.164664 11428 tablet_service.cc:1511] Processing CreateTablet for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:14:20.166023 11428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.100236 11560 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.108373 11561 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.169739 11357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99667fcf16014dc1b9b174f0a37d16af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.170151 11518 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.193749 11590 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.171105 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d79fba792428db7fcc1994fff81f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.136662 11570 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:20.197196 11570 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:20.197956 11570 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.200114 11561 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.202277 11560 ts_tablet_manager.cc:1434] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.178s	user 0.014s	sys 0.001s
I20260709 13:14:20.177950 11559 tablet_bootstrap.cc:654] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.216388 11519 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:20.217446 11408 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:20.220037 11360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ea18eac305644ed8d44618c2c0a8d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.221128 11518 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:20.211467 11560 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.243527 11561 ts_tablet_manager.cc:1434] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.224s	user 0.011s	sys 0.009s
I20260709 13:14:20.244236 11592 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.244386 11503 tablet_service.cc:1511] Processing CreateTablet for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:14:20.244735 11561 tablet_bootstrap.cc:492] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:20.244710 11592 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.245656 11503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.246848 11592 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.247984 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:20.248286 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:14:20.249315 11333 leader_election.cc:343] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:20.249745 11335 leader_election.cc:343] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.250161 11335 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:20.251176 11592 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:20.255373 11561 tablet_bootstrap.cc:654] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.263671 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:20.272025 11593 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:20.272468 11593 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:20.273229 11593 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.282809 11504 tablet_service.cc:1511] Processing CreateTablet for tablet b8ec7d9e61ab4ff5bd59a37021d68dba (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:14:20.235558 11565 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.283183 11354 tablet_service.cc:1511] Processing CreateTablet for tablet b8ec7d9e61ab4ff5bd59a37021d68dba (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:14:20.284102 11504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ec7d9e61ab4ff5bd59a37021d68dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.284314 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ec7d9e61ab4ff5bd59a37021d68dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.290235 11352 tablet_service.cc:1511] Processing CreateTablet for tablet a38969de5c824718aa81b9622624bc51 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:14:20.291380 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38969de5c824718aa81b9622624bc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.293567 11507 tablet_service.cc:1511] Processing CreateTablet for tablet a38969de5c824718aa81b9622624bc51 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:14:20.294728 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38969de5c824718aa81b9622624bc51. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:20.299716 11409 leader_election.cc:343] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.310633 11565 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.330494 11565 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.337030 11354 tablet_service.cc:1511] Processing CreateTablet for tablet 78b9049e00f94cc1912970768037cf24 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:14:20.337275 11352 tablet_service.cc:1511] Processing CreateTablet for tablet 8a142ac56caa42a08e0a838dc87ac3b3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:14:20.321540 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 104184us
I20260709 13:14:20.338207 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b9049e00f94cc1912970768037cf24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.336352 11560 tablet_bootstrap.cc:654] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.344404 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:20.360646 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:20.361846 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 78b9049e00f94cc1912970768037cf24 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 13:14:20.362156 11409 leader_election.cc:343] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.363051 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b9049e00f94cc1912970768037cf24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.364388 11505 tablet_service.cc:1511] Processing CreateTablet for tablet 8a142ac56caa42a08e0a838dc87ac3b3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:14:20.365173 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad5a541019548a3b543619a92b420b1 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:14:20.366346 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad5a541019548a3b543619a92b420b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:20.369275 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:20.379246 11268 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.382422 11565 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.383872 11433 tablet_service.cc:1511] Processing CreateTablet for tablet 78b9049e00f94cc1912970768037cf24 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:14:20.385151 11433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b9049e00f94cc1912970768037cf24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.387454 11352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a142ac56caa42a08e0a838dc87ac3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.389513 11428 tablet_service.cc:1511] Processing CreateTablet for tablet b8ec7d9e61ab4ff5bd59a37021d68dba (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:14:20.390693 11428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ec7d9e61ab4ff5bd59a37021d68dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.392709 11426 tablet_service.cc:1511] Processing CreateTablet for tablet 8a142ac56caa42a08e0a838dc87ac3b3 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:14:20.393142 11408 leader_election.cc:343] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.393584 11408 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:20.393980 11426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a142ac56caa42a08e0a838dc87ac3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.365648 11505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a142ac56caa42a08e0a838dc87ac3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.396203 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:20.397608 11430 tablet_service.cc:1511] Processing CreateTablet for tablet a38969de5c824718aa81b9622624bc51 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:14:20.398864 11430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38969de5c824718aa81b9622624bc51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.400936 11354 tablet_service.cc:1511] Processing CreateTablet for tablet e6e7edf5f82e49158a4023ea61d78284 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:14:20.402030 11354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e7edf5f82e49158a4023ea61d78284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.402200 11432 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad5a541019548a3b543619a92b420b1 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:14:20.403995 11432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad5a541019548a3b543619a92b420b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.406543 11507 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad5a541019548a3b543619a92b420b1 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:14:20.407905 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad5a541019548a3b543619a92b420b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.410645 11565 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:20.439502 11353 tablet_service.cc:1511] Processing CreateTablet for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:14:20.440285 11561 tablet_bootstrap.cc:492] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:20.440902 11561 ts_tablet_manager.cc:1403] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.196s	user 0.025s	sys 0.025s
I20260709 13:14:20.440932 11353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:20.476121 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 83021.3us
W20260709 13:14:20.495744 11408 leader_election.cc:343] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.496194 11408 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:20.497103 11507 tablet_service.cc:1511] Processing CreateTablet for tablet e6e7edf5f82e49158a4023ea61d78284 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:14:20.497481 11427 tablet_service.cc:1511] Processing CreateTablet for tablet e6e7edf5f82e49158a4023ea61d78284 (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:14:20.498236 11507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e7edf5f82e49158a4023ea61d78284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.498520 11427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e7edf5f82e49158a4023ea61d78284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.499085 11560 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.498739 11561 raft_consensus.cc:359] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.500092 11560 ts_tablet_manager.cc:1403] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.289s	user 0.020s	sys 0.004s
I20260709 13:14:20.504364 11560 raft_consensus.cc:359] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.505012 11560 raft_consensus.cc:385] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.505280 11560 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.505898 11593 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:20.505816 11434 tablet_service.cc:1511] Processing CreateTablet for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:14:20.506105 11560 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.507033 11434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.508417 11566 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.509583 11560 ts_tablet_manager.cc:1434] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 13:14:20.510815 11560 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.512036 11506 tablet_service.cc:1511] Processing CreateTablet for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae (DEFAULT_TABLE table=test-workload [id=860deb7e587340d8a0da392b46ada4ae]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:14:20.513247 11506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:20.516208 11559 tablet_bootstrap.cc:492] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:20.516698 11559 ts_tablet_manager.cc:1403] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.358s	user 0.030s	sys 0.031s
I20260709 13:14:20.517623 11560 tablet_bootstrap.cc:654] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.500411 11561 raft_consensus.cc:385] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.519340 11559 raft_consensus.cc:359] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.544885 11559 raft_consensus.cc:385] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.545226 11559 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.545965 11559 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.534691 11561 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.556432 11561 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.559157 11561 ts_tablet_manager.cc:1434] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.118s	user 0.006s	sys 0.004s
I20260709 13:14:20.560273 11561 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:20.564327 11559 ts_tablet_manager.cc:1434] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.003s
I20260709 13:14:20.565301 11559 tablet_bootstrap.cc:492] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:20.574165 11561 tablet_bootstrap.cc:654] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.576999 11267 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.578070 11559 tablet_bootstrap.cc:654] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.588625 11560 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.584340 11566 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.589717 11560 ts_tablet_manager.cc:1403] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.025s
I20260709 13:14:20.592960 11566 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.594161 11560 raft_consensus.cc:359] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.594977 11560 raft_consensus.cc:385] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.595317 11560 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.596474 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:20.598582 11483 leader_election.cc:343] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.603385 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:20.604094 11445 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:20.606642 11561 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:20.607105 11561 ts_tablet_manager.cc:1403] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.017s
I20260709 13:14:20.607203 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:20.608564 11605 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:20.608901 11605 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.609198 11605 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.611379 11570 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.600486 11560 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.617722 11560 ts_tablet_manager.cc:1434] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.002s
I20260709 13:14:20.618738 11560 tablet_bootstrap.cc:492] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.604902 11566 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:20.630658 11605 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.636229 11560 tablet_bootstrap.cc:654] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.636664 11559 tablet_bootstrap.cc:492] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:20.637130 11559 ts_tablet_manager.cc:1403] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.003s
I20260709 13:14:20.632124 11566 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:14:20.648715 11570 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.649181 11267 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:20.650473 11605 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:20.651927 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:20.651702 11561 raft_consensus.cc:359] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.652645 11561 raft_consensus.cc:385] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.654772 11561 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.655575 11570 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.654246 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:20.654089 11559 raft_consensus.cc:359] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.657303 11559 raft_consensus.cc:385] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:14:20.659961 11483 leader_election.cc:343] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.657701 11559 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.660013 11444 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.655599 11561 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.671380 11559 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.675341 11561 ts_tablet_manager.cc:1434] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 13:14:20.676780 11561 tablet_bootstrap.cc:492] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:20.675518 11559 ts_tablet_manager.cc:1434] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 13:14:20.681241 11559 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:20.685685 11444 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:20.687021 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:20.688386 11612 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:20.688865 11612 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:20.689678 11612 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.698364 11559 tablet_bootstrap.cc:654] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.712356 11592 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.712901 11592 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.714156 11560 tablet_bootstrap.cc:492] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.714625 11560 ts_tablet_manager.cc:1403] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.096s	user 0.014s	sys 0.018s
I20260709 13:14:20.716571 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:20.717121 11560 raft_consensus.cc:359] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.717732 11560 raft_consensus.cc:385] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:14:20.717927 11335 leader_election.cc:343] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.718844 11592 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.720293 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:20.723455 11333 leader_election.cc:343] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:20.723884 11333 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:20.724942 11592 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:20.717993 11560 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.727923 11560 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.729692 11561 tablet_bootstrap.cc:654] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.735833 11560 ts_tablet_manager.cc:1434] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:14:20.748848 11268 catalog_manager.cc:5697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:20.756534 11560 tablet_bootstrap.cc:492] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.759104 11443 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.769030 11559 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:20.769570 11559 ts_tablet_manager.cc:1403] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.004s
I20260709 13:14:20.771688 11592 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.772131 11559 raft_consensus.cc:359] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.772946 11559 raft_consensus.cc:385] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.773252 11559 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.774015 11559 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.775939 11559 ts_tablet_manager.cc:1434] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:20.776595 11560 tablet_bootstrap.cc:654] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.777009 11559 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:20.782764 11559 tablet_bootstrap.cc:654] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.785454 11561 tablet_bootstrap.cc:492] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:20.785573 11560 tablet_bootstrap.cc:492] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.785952 11561 ts_tablet_manager.cc:1403] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.109s	user 0.035s	sys 0.024s
I20260709 13:14:20.785959 11560 ts_tablet_manager.cc:1403] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.002s
I20260709 13:14:20.788894 11561 raft_consensus.cc:359] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.789573 11561 raft_consensus.cc:385] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.789852 11561 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.790522 11561 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.792766 11561 ts_tablet_manager.cc:1434] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:20.794104 11561 tablet_bootstrap.cc:492] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:20.791080 11560 raft_consensus.cc:359] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.798614 11560 raft_consensus.cc:385] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.798921 11560 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.799670 11560 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.800962 11559 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:20.801616 11559 ts_tablet_manager.cc:1403] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.006s
I20260709 13:14:20.801903 11560 ts_tablet_manager.cc:1434] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.000s
I20260709 13:14:20.802786 11560 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.804414 11559 raft_consensus.cc:359] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.805099 11559 raft_consensus.cc:385] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.805176 11561 tablet_bootstrap.cc:654] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.805509 11559 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.806232 11559 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.808243 11560 tablet_bootstrap.cc:654] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.808298 11559 ts_tablet_manager.cc:1434] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:20.809437 11559 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:20.814981 11559 tablet_bootstrap.cc:654] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.816881 11561 tablet_bootstrap.cc:492] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:20.817457 11561 ts_tablet_manager.cc:1403] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.006s
I20260709 13:14:20.818153 11560 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.818627 11560 ts_tablet_manager.cc:1403] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:14:20.820307 11561 raft_consensus.cc:359] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.821058 11561 raft_consensus.cc:385] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.821437 11561 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.821143 11560 raft_consensus.cc:359] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.821930 11560 raft_consensus.cc:385] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.822194 11560 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.822389 11561 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.822925 11560 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.825819 11560 ts_tablet_manager.cc:1434] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:14:20.827664 11561 ts_tablet_manager.cc:1434] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:14:20.828584 11561 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:20.829090 11560 tablet_bootstrap.cc:492] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.834019 11561 tablet_bootstrap.cc:654] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.834519 11560 tablet_bootstrap.cc:654] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.839537 11559 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:20.840003 11559 ts_tablet_manager.cc:1403] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.013s
I20260709 13:14:20.844604 11560 tablet_bootstrap.cc:492] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.845132 11560 ts_tablet_manager.cc:1403] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.016s	user 0.002s	sys 0.009s
I20260709 13:14:20.845007 11593 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:20.845821 11561 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:20.846230 11561 ts_tablet_manager.cc:1403] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.005s
I20260709 13:14:20.846727 11559 raft_consensus.cc:359] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.847340 11559 raft_consensus.cc:385] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.847633 11559 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.847956 11560 raft_consensus.cc:359] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.848240 11559 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.848624 11560 raft_consensus.cc:385] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.848686 11561 raft_consensus.cc:359] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.849006 11560 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.849345 11561 raft_consensus.cc:385] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.849656 11561 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.849787 11560 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.850350 11561 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.852311 11560 ts_tablet_manager.cc:1434] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:20.852365 11561 ts_tablet_manager.cc:1434] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:14:20.853309 11560 tablet_bootstrap.cc:492] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.855566 11561 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:20.859640 11560 tablet_bootstrap.cc:654] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.861199 11565 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.861733 11561 tablet_bootstrap.cc:654] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.861670 11565 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.863260 11559 ts_tablet_manager.cc:1434] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:14:20.863744 11565 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.864240 11559 tablet_bootstrap.cc:492] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:20.865486 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:20.866164 11370 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:20.867678 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:20.869647 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:20.870851 11518 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:20.872721 11565 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:20.873180 11565 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.873605 11565 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.873206 11560 tablet_bootstrap.cc:492] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.874327 11560 ts_tablet_manager.cc:1403] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.013s
I20260709 13:14:20.876585 11559 tablet_bootstrap.cc:654] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.877851 11560 raft_consensus.cc:359] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.878526 11560 raft_consensus.cc:385] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.878832 11560 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.879554 11560 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.885057 11566 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.885610 11566 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.888831 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:20.889683 11370 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:20.889953 11566 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:20.891248 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:20.892360 11566 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:20.892788 11566 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.893170 11566 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.913172 11565 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.917523 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:20.918270 11445 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:20.919910 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:20.915750 11560 ts_tablet_manager.cc:1434] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.020s
I20260709 13:14:20.921393 11560 tablet_bootstrap.cc:492] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.923035 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:20.923547 11370 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.924396 11519 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.925348 11565 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.930284 11561 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:20.930766 11561 ts_tablet_manager.cc:1403] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.006s
W20260709 13:14:20.931911 11409 consensus_peers.cc:597] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:20.933291 11560 tablet_bootstrap.cc:654] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.934852 11566 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.937585 11370 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:20.941195 11561 raft_consensus.cc:359] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.941668 11561 raft_consensus.cc:385] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.941874 11561 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.943243 11561 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.942478 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:20.945746 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:20.946285 11370 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.952607 11370 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:20.951167 11519 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:20.954667 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:20.955461 11612 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:20.955914 11612 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:20.957120 11561 ts_tablet_manager.cc:1434] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:14:20.956745 11612 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.956044 11641 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:20.958685 11561 tablet_bootstrap.cc:492] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:20.961161 11560 tablet_bootstrap.cc:492] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:20.958590 11641 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:20.961598 11560 ts_tablet_manager.cc:1403] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.040s	user 0.022s	sys 0.010s
I20260709 13:14:20.965111 11566 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:20.965879 11559 tablet_bootstrap.cc:492] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:20.966112 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:20.966661 11445 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.967998 11560 raft_consensus.cc:359] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.968627 11560 raft_consensus.cc:385] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.968869 11560 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.969483 11560 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.962229 11641 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.973573 11560 ts_tablet_manager.cc:1434] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.002s
I20260709 13:14:20.974468 11560 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:20.974275 11445 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:20.973889 11265 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:20.966320 11559 ts_tablet_manager.cc:1403] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.102s	user 0.014s	sys 0.013s
I20260709 13:14:20.977890 11592 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.978490 11561 tablet_bootstrap.cc:654] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.978358 11592 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.980414 11592 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.980705 11560 tablet_bootstrap.cc:654] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:20.982299 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:20.982887 11519 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:20.984061 11333 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:20.983995 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:20.984863 11592 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:20.985299 11592 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.985677 11592 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:20.988796 11559 raft_consensus.cc:359] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.989487 11559 raft_consensus.cc:385] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:20.989743 11559 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:20.990713 11559 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.993218 11592 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:20.995801 11592 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:20.984858 11445 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:20.996543 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:20.997221 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:20.997740 11519 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.000198 11612 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.000656 11612 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.001816 11561 tablet_bootstrap.cc:492] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.003278 11566 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.003541 11561 ts_tablet_manager.cc:1403] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.011s
I20260709 13:14:21.003728 11566 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.004300 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.004892 11445 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.005661 11566 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.006229 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:21.006801 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.007378 11370 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.007776 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.007599 11561 raft_consensus.cc:359] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.008212 11641 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.008478 11561 raft_consensus.cc:385] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.008903 11612 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.008978 11561 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.008559 11566 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.009644 11566 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.009939 11566 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.009967 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.010257 11561 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:21.011492 11483 leader_election.cc:343] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.012593 11561 ts_tablet_manager.cc:1434] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:14:21.013862 11561 tablet_bootstrap.cc:492] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.015079 11560 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.015569 11560 ts_tablet_manager.cc:1403] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.004s
I20260709 13:14:21.013885 11612 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.016330 11612 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.017930 11559 ts_tablet_manager.cc:1434] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.000s
I20260709 13:14:21.018285 11560 raft_consensus.cc:359] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.016405 11566 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.018987 11560 raft_consensus.cc:385] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.019229 11561 tablet_bootstrap.cc:654] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.019373 11560 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.020291 11560 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.021901 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.022467 11445 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.022560 11560 ts_tablet_manager.cc:1434] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:14:21.023514 11560 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.025405 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:21.025969 11369 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.029024 11560 tablet_bootstrap.cc:654] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.035468 11369 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:21.036839 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.037734 11605 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.038230 11605 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:21.039080 11605 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.019593 11559 tablet_bootstrap.cc:492] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.019186 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.045081 11370 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.046523 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.008661 11641 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.047896 11614 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.048713 11519 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.049968 11612 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.067070 11445 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:21.068271 11592 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.068761 11592 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.074492 11559 tablet_bootstrap.cc:654] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.076150 11333 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:14:21.077126 11590 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.075204 11443 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.077694 11590 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:21.078688 11590 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.079828 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:21.081243 11612 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.081687 11612 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.082038 11612 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.085009 11592 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.087246 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.087496 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.088277 11443 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:21.087277 11641 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.046811 11265 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.088541 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.088304 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.089610 11445 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.090054 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.090417 11335 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:21.049422 11566 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.091085 11614 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.094808 11592 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.098390 11614 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:21.096529 11409 leader_election.cc:343] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.112949 11614 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.128077 11596 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.128672 11596 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.130906 11596 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.131872 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.132853 11268 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:21.136171 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.137701 11333 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:21.141144 11566 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.141582 11590 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.115602 11266 catalog_manager.cc:5697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.155556 11612 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.169409 11518 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:21.170059 11561 tablet_bootstrap.cc:492] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.170506 11561 ts_tablet_manager.cc:1403] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.157s	user 0.072s	sys 0.073s
I20260709 13:14:21.171372 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.172951 11641 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.176919 11561 raft_consensus.cc:359] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.177726 11561 raft_consensus.cc:385] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.177994 11561 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.178823 11641 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.178872 11561 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.180521 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:21.181111 11369 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.181699 11592 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.183960 11561 ts_tablet_manager.cc:1434] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:14:21.184952 11592 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.185428 11592 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.186851 11612 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.188670 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.189152 11560 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.190155 11561 tablet_bootstrap.cc:492] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.190266 11560 ts_tablet_manager.cc:1403] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.167s	user 0.038s	sys 0.066s
I20260709 13:14:21.191859 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:21.189874 11592 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.195921 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.191211 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.203033 11443 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
W20260709 13:14:21.204232 11333 leader_election.cc:343] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.205505 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.204160 11566 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:14:21.193315 11560 raft_consensus.cc:359] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.207432 11560 raft_consensus.cc:385] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.207702 11560 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.208374 11560 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:21.209307 11483 leader_election.cc:343] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.210400 11560 ts_tablet_manager.cc:1434] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 13:14:21.211759 11592 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.211881 11561 tablet_bootstrap.cc:654] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.211301 11560 tablet_bootstrap.cc:492] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.189127 11369 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:21.214951 11559 tablet_bootstrap.cc:492] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.215378 11559 ts_tablet_manager.cc:1403] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.196s	user 0.021s	sys 0.012s
I20260709 13:14:21.216318 11614 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.212183 11592 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.217747 11592 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.207056 11444 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:21.218822 11559 raft_consensus.cc:359] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.220027 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.220235 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:21.220881 11566 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.219503 11559 raft_consensus.cc:385] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.221341 11566 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:21.221547 11559 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.221769 11613 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.222358 11613 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:21.222740 11559 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.223179 11613 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.224934 11559 ts_tablet_manager.cc:1434] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.001s
I20260709 13:14:21.222414 11566 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.225944 11559 tablet_bootstrap.cc:492] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.229365 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.227792 11641 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.238656 11559 tablet_bootstrap.cc:654] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.235806 11445 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.244144 11643 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.236186 11641 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.245378 11643 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.252851 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.252956 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:21.254302 11409 leader_election.cc:343] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:21.254848 11408 leader_election.cc:343] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.255244 11408 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.255697 11643 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.255973 11565 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.257277 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:21.258625 11409 leader_election.cc:343] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.261165 11561 tablet_bootstrap.cc:492] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.261647 11561 ts_tablet_manager.cc:1403] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.014s
I20260709 13:14:21.265034 11559 tablet_bootstrap.cc:492] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.265085 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.265650 11641 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.258636 11560 tablet_bootstrap.cc:654] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:21.266642 11408 leader_election.cc:343] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.267104 11408 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.267324 11559 ts_tablet_manager.cc:1403] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.009s
I20260709 13:14:21.268395 11614 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.268919 11641 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.268885 11614 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.271055 11614 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.271719 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.272424 11368 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.273365 11561 raft_consensus.cc:359] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.273504 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.273823 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.274286 11444 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.274737 11445 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:21.274878 11559 raft_consensus.cc:359] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.275588 11559 raft_consensus.cc:385] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.275854 11559 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.276575 11614 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.276991 11614 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.276779 11559 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.277416 11614 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.263216 11592 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.278640 11559 ts_tablet_manager.cc:1434] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:14:21.279445 11559 tablet_bootstrap.cc:492] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.282035 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:21.282663 11518 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.282585 11561 raft_consensus.cc:385] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.283507 11561 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.284588 11561 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.287742 11561 ts_tablet_manager.cc:1434] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:14:21.287823 11268 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:21.290072 11614 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.290447 11268 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:21.290582 11561 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.291679 11518 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.292798 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.293198 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:21.293612 11590 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.293737 11368 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.294091 11590 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:21.294916 11590 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.300748 11560 tablet_bootstrap.cc:492] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.301239 11560 ts_tablet_manager.cc:1403] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.000s
I20260709 13:14:21.301433 11368 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:21.302616 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.303548 11613 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.304090 11613 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:21.305454 11561 tablet_bootstrap.cc:654] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.305025 11613 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.305840 11592 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.309015 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.309588 11560 raft_consensus.cc:359] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.310344 11560 raft_consensus.cc:385] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.310618 11560 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.311713 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.312299 11444 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.314949 11560 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.319809 11444 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:21.321476 11614 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.309653 11445 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.323279 11559 tablet_bootstrap.cc:654] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.328745 11560 ts_tablet_manager.cc:1434] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:14:21.329861 11560 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.331799 11561 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.332237 11561 ts_tablet_manager.cc:1403] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.004s
I20260709 13:14:21.337327 11265 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:21.345686 11596 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.347532 11596 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.350088 11596 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.349815 11561 raft_consensus.cc:359] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.350807 11561 raft_consensus.cc:385] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.351068 11561 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.351805 11561 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.352083 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.353725 11561 ts_tablet_manager.cc:1434] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
W20260709 13:14:21.353652 11333 leader_election.cc:343] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.354636 11561 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.352133 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.355255 11444 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:21.356488 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.357249 11590 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.357678 11590 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.357908 11445 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.358191 11590 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.359380 11268 catalog_manager.cc:5697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.360495 11561 tablet_bootstrap.cc:654] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.366019 11559 tablet_bootstrap.cc:492] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.366514 11559 ts_tablet_manager.cc:1403] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.087s	user 0.022s	sys 0.005s
I20260709 13:14:21.366853 11560 tablet_bootstrap.cc:654] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.366741 11641 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.369695 11559 raft_consensus.cc:359] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.370496 11559 raft_consensus.cc:385] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.370795 11559 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.370909 11614 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.378333 11559 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.384752 11559 ts_tablet_manager.cc:1434] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.003s
I20260709 13:14:21.385857 11559 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.391044 11641 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:21.391201 11590 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.405876 11561 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.406744 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.406355 11561 ts_tablet_manager.cc:1403] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.021s
I20260709 13:14:21.408377 11559 tablet_bootstrap.cc:654] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.407308 11445 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:21.415081 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:21.417393 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:21.418051 11519 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.419677 11445 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.420768 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:21.421779 11596 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.422228 11596 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:21.422994 11596 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.415444 11590 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.432955 11561 raft_consensus.cc:359] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.433727 11561 raft_consensus.cc:385] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.434036 11561 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.434720 11561 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.436807 11561 ts_tablet_manager.cc:1434] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:14:21.437742 11561 tablet_bootstrap.cc:492] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.445724 11519 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.447057 11560 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.447494 11560 ts_tablet_manager.cc:1403] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.118s	user 0.017s	sys 0.011s
I20260709 13:14:21.445747 11614 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:14:21.452129 11561 tablet_bootstrap.cc:654] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.452369 11560 raft_consensus.cc:359] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.452986 11560 raft_consensus.cc:385] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.453230 11560 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.455056 11369 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.459218 11560 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.461558 11560 ts_tablet_manager.cc:1434] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:14:21.462687 11560 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.466172 11559 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.466698 11559 ts_tablet_manager.cc:1403] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.081s	user 0.006s	sys 0.005s
I20260709 13:14:21.468698 11560 tablet_bootstrap.cc:654] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.469939 11559 raft_consensus.cc:359] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.470677 11559 raft_consensus.cc:385] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.470937 11559 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.472908 11265 catalog_manager.cc:5697] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.476830 11559 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.478976 11559 ts_tablet_manager.cc:1434] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:14:21.480837 11560 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.481280 11560 ts_tablet_manager.cc:1403] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.001s
I20260709 13:14:21.484952 11559 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.485917 11561 tablet_bootstrap.cc:492] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.486469 11561 ts_tablet_manager.cc:1403] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.049s	user 0.020s	sys 0.014s
I20260709 13:14:21.489195 11561 raft_consensus.cc:359] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.490000 11561 raft_consensus.cc:385] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.490357 11561 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.490691 11559 tablet_bootstrap.cc:654] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.491151 11561 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.494616 11560 raft_consensus.cc:359] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.495271 11560 raft_consensus.cc:385] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.495553 11560 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.496181 11560 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.498004 11560 ts_tablet_manager.cc:1434] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.001s
I20260709 13:14:21.498642 11560 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.498934 11561 ts_tablet_manager.cc:1434] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:14:21.499908 11561 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.505137 11560 tablet_bootstrap.cc:654] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.505184 11561 tablet_bootstrap.cc:654] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.516695 11561 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.517419 11561 ts_tablet_manager.cc:1403] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.002s
I20260709 13:14:21.518432 11560 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.518900 11560 ts_tablet_manager.cc:1403] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.009s
I20260709 13:14:21.519182 11559 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.519624 11559 ts_tablet_manager.cc:1403] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.008s
I20260709 13:14:21.520087 11561 raft_consensus.cc:359] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.520756 11561 raft_consensus.cc:385] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.521027 11561 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.521687 11560 raft_consensus.cc:359] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.522569 11560 raft_consensus.cc:385] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.522315 11559 raft_consensus.cc:359] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.522967 11559 raft_consensus.cc:385] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.521737 11561 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.523319 11559 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.522879 11560 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.524273 11560 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.525144 11561 ts_tablet_manager.cc:1434] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:21.526105 11561 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.526701 11560 ts_tablet_manager.cc:1434] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 13:14:21.527595 11560 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.531750 11596 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.532768 11561 tablet_bootstrap.cc:654] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.532928 11560 tablet_bootstrap.cc:654] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.526028 11559 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.538357 11559 ts_tablet_manager.cc:1434] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:14:21.539256 11559 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.547915 11559 tablet_bootstrap.cc:654] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.548424 11566 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:21.559692 11641 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.560330 11641 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.564913 11641 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.565198 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.566605 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:21.568200 11409 leader_election.cc:343] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:21.569406 11408 leader_election.cc:343] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.569885 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.571013 11641 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.578784 11641 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.579329 11641 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.581573 11641 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.583660 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.584790 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:21.586238 11408 leader_election.cc:343] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.587342 11596 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.587888 11596 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.590065 11596 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:14:21.592067 11409 leader_election.cc:343] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.592542 11409 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.593766 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.594412 11518 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:21.595640 11333 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.595811 11641 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.596932 11596 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.597324 11596 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.597134 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.597857 11444 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:21.599545 11596 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.614064 11566 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.615723 11445 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.623427 11596 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.626416 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.626978 11444 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.634135 11444 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.635514 11335 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:21.636268 11592 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.639901 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:21.640519 11519 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.644243 11592 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:21.645150 11592 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.646926 11519 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.648810 11596 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.655084 11560 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.655575 11560 ts_tablet_manager.cc:1403] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.128s	user 0.056s	sys 0.067s
I20260709 13:14:21.655738 11561 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.656136 11561 ts_tablet_manager.cc:1403] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.130s	user 0.030s	sys 0.021s
I20260709 13:14:21.657096 11592 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.658164 11559 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.658041 11560 raft_consensus.cc:359] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.658704 11559 ts_tablet_manager.cc:1403] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.120s	user 0.026s	sys 0.025s
I20260709 13:14:21.658712 11560 raft_consensus.cc:385] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.659181 11560 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.660146 11560 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.661583 11559 raft_consensus.cc:359] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.662082 11560 ts_tablet_manager.cc:1434] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:21.662302 11559 raft_consensus.cc:385] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.662675 11559 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.663110 11560 tablet_bootstrap.cc:492] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.663388 11559 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.665319 11559 ts_tablet_manager.cc:1434] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:14:21.666320 11559 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.668762 11560 tablet_bootstrap.cc:654] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.672821 11561 raft_consensus.cc:359] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.673491 11561 raft_consensus.cc:385] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.673753 11561 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.674363 11561 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.679103 11561 ts_tablet_manager.cc:1434] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:14:21.680037 11561 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.684158 11559 tablet_bootstrap.cc:654] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.684562 11592 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.685139 11592 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.690419 11592 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.691955 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.691970 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.696165 11445 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:21.697413 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:21.699275 11596 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.698021 11267 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.699903 11560 tablet_bootstrap.cc:492] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.700393 11560 ts_tablet_manager.cc:1403] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.007s
I20260709 13:14:21.701215 11687 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.701653 11687 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.702790 11560 raft_consensus.cc:359] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.703424 11560 raft_consensus.cc:385] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.703693 11687 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.703779 11560 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.704562 11560 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.704981 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.705565 11519 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:21.706564 11560 ts_tablet_manager.cc:1434] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:21.707470 11560 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.707463 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.708140 11445 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:21.708259 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.709695 11590 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.710035 11590 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.710352 11590 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.711102 11559 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.711549 11559 ts_tablet_manager.cc:1403] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.007s
I20260709 13:14:21.713511 11560 tablet_bootstrap.cc:654] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.714330 11559 raft_consensus.cc:359] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.715008 11559 raft_consensus.cc:385] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.715266 11559 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.716106 11559 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.720592 11596 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.721410 11590 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.724169 11590 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.722267 11559 ts_tablet_manager.cc:1434] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:14:21.724191 11566 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:21.725441 11559 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.726881 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.727458 11445 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.732303 11561 tablet_bootstrap.cc:654] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.736043 11559 tablet_bootstrap.cc:654] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.738390 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:21.738936 11519 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.741752 11560 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.742237 11560 ts_tablet_manager.cc:1403] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.035s	user 0.028s	sys 0.004s
I20260709 13:14:21.744840 11560 raft_consensus.cc:359] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.745484 11560 raft_consensus.cc:385] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.745577 11592 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.745754 11560 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.746305 11592 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.746587 11560 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.752105 11561 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.752451 11560 ts_tablet_manager.cc:1434] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:14:21.752526 11561 ts_tablet_manager.cc:1403] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.006s
I20260709 13:14:21.753345 11560 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.754774 11612 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.754951 11561 raft_consensus.cc:359] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.755620 11561 raft_consensus.cc:385] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.755905 11561 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.756497 11561 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.759563 11561 ts_tablet_manager.cc:1434] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:21.759711 11560 tablet_bootstrap.cc:654] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.760430 11561 tablet_bootstrap.cc:492] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.772342 11561 tablet_bootstrap.cc:654] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.775913 11612 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.777273 11612 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.778379 11560 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.778857 11560 ts_tablet_manager.cc:1403] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 13:14:21.781629 11560 raft_consensus.cc:359] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.782294 11560 raft_consensus.cc:385] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.782534 11560 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.783182 11560 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.785147 11560 ts_tablet_manager.cc:1434] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:21.786149 11560 tablet_bootstrap.cc:492] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.785181 11561 tablet_bootstrap.cc:492] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.786651 11561 ts_tablet_manager.cc:1403] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.004s
I20260709 13:14:21.789127 11561 raft_consensus.cc:359] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.791661 11560 tablet_bootstrap.cc:654] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.792375 11561 raft_consensus.cc:385] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.792714 11561 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.793718 11561 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.794747 11559 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.795140 11559 ts_tablet_manager.cc:1403] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.070s	user 0.011s	sys 0.017s
I20260709 13:14:21.795957 11561 ts_tablet_manager.cc:1434] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:14:21.796811 11561 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.797588 11559 raft_consensus.cc:359] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.798753 11559 raft_consensus.cc:385] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.799001 11559 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.799649 11559 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.802325 11559 ts_tablet_manager.cc:1434] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:21.803359 11559 tablet_bootstrap.cc:492] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.804556 11592 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.805042 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.805426 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.805662 11442 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:21.806121 11518 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:21.807291 11335 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:21.808264 11590 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.808867 11559 tablet_bootstrap.cc:654] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.809154 11592 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.810006 11519 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.812832 11561 tablet_bootstrap.cc:654] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.816681 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.812876 11445 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.809514 11592 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.817898 11590 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.818171 11592 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.818778 11596 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.820288 11590 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:21.825145 11614 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.829187 11590 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.842329 11612 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.846275 11565 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.846781 11565 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.848784 11565 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.848521 11592 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.851517 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.851727 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.852154 11369 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
W20260709 13:14:21.853176 11408 leader_election.cc:343] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.853746 11409 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.853996 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:21.854579 11517 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.854869 11565 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.854725 11592 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.855258 11565 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.855659 11565 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.853322 11560 tablet_bootstrap.cc:492] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:21.857214 11560 ts_tablet_manager.cc:1403] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.004s
I20260709 13:14:21.860109 11560 raft_consensus.cc:359] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.860816 11560 raft_consensus.cc:385] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.861083 11560 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.863216 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:21.863754 11443 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.871049 11641 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.871572 11641 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.873739 11641 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.874468 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:21.876184 11409 leader_election.cc:343] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.878855 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:21.880440 11561 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:21.880923 11561 ts_tablet_manager.cc:1403] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.000s
I20260709 13:14:21.881705 11560 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.884718 11560 ts_tablet_manager.cc:1434] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:14:21.885697 11560 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:21.889281 11561 raft_consensus.cc:359] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.889973 11561 raft_consensus.cc:385] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.890236 11561 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.899839 11559 tablet_bootstrap.cc:492] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:21.900344 11559 ts_tablet_manager.cc:1403] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.097s	user 0.023s	sys 0.037s
W20260709 13:14:21.902459 11408 leader_election.cc:343] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.902921 11408 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:21.903661 11641 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:21.904806 11443 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.898483 11517 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:21.901046 11561 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.909503 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:21.910748 11687 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.911213 11687 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:21.912231 11561 ts_tablet_manager.cc:1434] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.003s
I20260709 13:14:21.912031 11687 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.913113 11561 tablet_bootstrap.cc:492] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:21.927492 11559 raft_consensus.cc:359] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.936983 11559 raft_consensus.cc:385] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:21.937394 11559 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:21.923383 11565 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.939464 11559 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.943142 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:21.943742 11368 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.948746 11560 tablet_bootstrap.cc:654] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.953313 11559 ts_tablet_manager.cc:1434] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.002s
I20260709 13:14:21.954349 11559 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:21.955003 11566 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.955498 11566 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.957120 11565 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:21.957440 11566 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:21.958329 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:21.958418 11687 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.959036 11369 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.960664 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:21.959223 11267 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:21.961416 11517 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.961663 11368 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:21.962625 11565 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.962954 11409 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:21.960670 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:21.964983 11710 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:21.965513 11710 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:21.966322 11710 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.963701 11687 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.964043 11442 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:21.968732 11484 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:21.969580 11517 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:21.971858 11566 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:21.972250 11566 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.963709 11565 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.972637 11566 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:21.972361 11592 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:21.974759 11559 tablet_bootstrap.cc:654] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:21.975745 11687 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:21.978008 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:21.999912 11409 leader_election.cc:343] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.000133 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.000254 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.000764 11442 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
W20260709 13:14:22.001626 11408 leader_election.cc:343] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:21.998812 11565 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.002633 11561 tablet_bootstrap.cc:654] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.002043 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.002022 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.004689 11565 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.007943 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.008522 11517 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:22.008767 11566 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.011734 11559 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.012313 11559 ts_tablet_manager.cc:1403] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.017s
I20260709 13:14:22.014484 11592 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.014842 11592 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.015147 11592 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.020588 11559 raft_consensus.cc:359] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.021250 11559 raft_consensus.cc:385] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.021219 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.021628 11559 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.021857 11368 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.022360 11559 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.024477 11559 ts_tablet_manager.cc:1434] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:14:22.025314 11559 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.028688 11592 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.031837 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.032002 11592 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.032224 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:22.032513 11442 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.032907 11517 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.034917 11560 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.018082 11687 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.035634 11560 ts_tablet_manager.cc:1403] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.150s	user 0.014s	sys 0.013s
I20260709 13:14:22.034022 11368 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.050002 11442 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:22.053467 11560 raft_consensus.cc:359] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.054562 11560 raft_consensus.cc:385] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.053925 11596 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.054978 11560 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.056010 11560 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.057809 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:22.058581 11560 ts_tablet_manager.cc:1434] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:14:22.058717 11614 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.059188 11614 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:22.059826 11560 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.060221 11614 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.063892 11517 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:22.062924 11267 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.066223 11566 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.065238 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:22.071717 11592 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.072249 11592 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:22.073087 11592 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.070169 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.075007 11443 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.077487 11565 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.078361 11565 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.080721 11565 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.085960 11561 tablet_bootstrap.cc:492] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:22.086383 11561 ts_tablet_manager.cc:1403] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.173s	user 0.009s	sys 0.014s
I20260709 13:14:22.086581 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.087144 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:22.088528 11408 leader_election.cc:343] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:22.089603 11409 leader_election.cc:343] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.090030 11409 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.091558 11443 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.091741 11565 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.096349 11267 catalog_manager.cc:5697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.096807 11561 raft_consensus.cc:359] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.097540 11561 raft_consensus.cc:385] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.097832 11561 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.098469 11561 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.100531 11561 ts_tablet_manager.cc:1434] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:14:22.101444 11561 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:22.102586 11559 tablet_bootstrap.cc:654] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.107992 11560 tablet_bootstrap.cc:654] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.114761 11561 tablet_bootstrap.cc:654] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.120931 11596 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.123849 11264 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:22.135725 11560 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.136209 11560 ts_tablet_manager.cc:1403] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.007s
I20260709 13:14:22.136857 11710 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.137331 11710 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.137494 11267 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.138798 11560 raft_consensus.cc:359] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.139425 11710 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.139458 11560 raft_consensus.cc:385] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.139855 11560 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.140193 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.140478 11560 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.141103 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:22.141541 11409 leader_election.cc:343] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:22.142387 11408 leader_election.cc:343] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.142727 11560 ts_tablet_manager.cc:1434] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:22.142784 11408 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.143745 11560 tablet_bootstrap.cc:492] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.143877 11710 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.145635 11559 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.149783 11560 tablet_bootstrap.cc:654] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.146095 11559 ts_tablet_manager.cc:1403] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.121s	user 0.013s	sys 0.013s
I20260709 13:14:22.153409 11559 raft_consensus.cc:359] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.154215 11559 raft_consensus.cc:385] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.154510 11559 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.155253 11559 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.157752 11559 ts_tablet_manager.cc:1434] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:22.159746 11559 tablet_bootstrap.cc:492] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.161828 11560 tablet_bootstrap.cc:492] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.162294 11560 ts_tablet_manager.cc:1403] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 13:14:22.164813 11560 raft_consensus.cc:359] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.165428 11560 raft_consensus.cc:385] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.165716 11559 tablet_bootstrap.cc:654] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.165696 11560 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.167066 11560 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.169407 11560 ts_tablet_manager.cc:1434] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:22.170348 11560 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.177631 11561 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:22.178092 11561 ts_tablet_manager.cc:1403] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.009s
I20260709 13:14:22.179051 11559 tablet_bootstrap.cc:492] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.179548 11559 ts_tablet_manager.cc:1403] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:14:22.180787 11561 raft_consensus.cc:359] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.181471 11561 raft_consensus.cc:385] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.181737 11561 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.182029 11559 raft_consensus.cc:359] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.182382 11561 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.182897 11559 raft_consensus.cc:385] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.183169 11559 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.183794 11559 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.184458 11561 ts_tablet_manager.cc:1434] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:22.185611 11561 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:22.185717 11559 ts_tablet_manager.cc:1434] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:14:22.186942 11560 tablet_bootstrap.cc:654] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.192411 11561 tablet_bootstrap.cc:654] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.196389 11559 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.197813 11560 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.198254 11560 ts_tablet_manager.cc:1403] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.003s
I20260709 13:14:22.201017 11560 raft_consensus.cc:359] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.201740 11560 raft_consensus.cc:385] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.202003 11560 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.203138 11559 tablet_bootstrap.cc:654] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.202926 11560 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.205525 11560 ts_tablet_manager.cc:1434] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:14:22.207208 11560 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.209615 11559 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.210147 11559 ts_tablet_manager.cc:1403] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.002s
I20260709 13:14:22.213347 11560 tablet_bootstrap.cc:654] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.213045 11559 raft_consensus.cc:359] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.213770 11559 raft_consensus.cc:385] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.214071 11559 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.214766 11559 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.216853 11559 ts_tablet_manager.cc:1434] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:14:22.217877 11559 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.225518 11559 tablet_bootstrap.cc:654] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.231788 11560 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.232309 11560 ts_tablet_manager.cc:1403] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.025s	user 0.023s	sys 0.000s
I20260709 13:14:22.233531 11561 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:22.234086 11561 ts_tablet_manager.cc:1403] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.007s
I20260709 13:14:22.234223 11559 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.234643 11559 ts_tablet_manager.cc:1403] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.002s
I20260709 13:14:22.235180 11560 raft_consensus.cc:359] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.235852 11560 raft_consensus.cc:385] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.236137 11560 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.236876 11560 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.237107 11559 raft_consensus.cc:359] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.237763 11559 raft_consensus.cc:385] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.238055 11559 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.238929 11559 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.239763 11560 ts_tablet_manager.cc:1434] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:22.241236 11560 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.241286 11559 ts_tablet_manager.cc:1434] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:22.242259 11559 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.247130 11561 raft_consensus.cc:359] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.247887 11561 raft_consensus.cc:385] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.248292 11560 tablet_bootstrap.cc:654] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.248174 11561 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.248067 11559 tablet_bootstrap.cc:654] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.249091 11561 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.252002 11710 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.252460 11710 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.254420 11565 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.254563 11592 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.254968 11565 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.255949 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.255039 11592 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.257179 11565 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:14:22.257665 11408 leader_election.cc:343] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.256053 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.258134 11592 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.259219 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:22.259596 11409 leader_election.cc:343] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.259822 11519 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:22.254400 11710 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.260648 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.261318 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.261425 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.262001 11444 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:22.262163 11710 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.262514 11710 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.262840 11710 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.260107 11409 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.263219 11335 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:14:22.264299 11409 leader_election.cc:343] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.264478 11592 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.264873 11592 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.265319 11565 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.265215 11592 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.266266 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.266649 11561 ts_tablet_manager.cc:1434] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:14:22.266850 11519 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:22.273571 11592 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.276243 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.276819 11444 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.281574 11710 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.284387 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.285317 11592 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.286983 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:22.287348 11710 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.287802 11519 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.288895 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:22.289489 11517 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.293197 11444 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:22.300235 11565 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.314010 11565 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.298254 11517 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
W20260709 13:14:22.296562 11409 leader_election.cc:343] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.316599 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:22.318123 11335 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.318480 11643 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.300470 11561 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:22.318938 11643 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:22.320492 11643 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.322994 11592 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.324244 11592 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:22.325073 11592 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.327152 11565 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.327502 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:22.328814 11409 leader_election.cc:343] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.330399 11561 tablet_bootstrap.cc:654] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.309420 11560 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.332442 11560 ts_tablet_manager.cc:1403] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.003s
I20260709 13:14:22.313336 11559 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.332507 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.295521 11519 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:22.333092 11559 ts_tablet_manager.cc:1403] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.091s	user 0.036s	sys 0.032s
I20260709 13:14:22.336650 11559 raft_consensus.cc:359] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.337325 11559 raft_consensus.cc:385] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.337603 11559 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
W20260709 13:14:22.337558 11408 leader_election.cc:343] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.338315 11408 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.339489 11710 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.339694 11559 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.341115 11560 raft_consensus.cc:359] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.341765 11560 raft_consensus.cc:385] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.342031 11560 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.346006 11559 ts_tablet_manager.cc:1434] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:14:22.346979 11559 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.347165 11560 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.348022 11561 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:22.348461 11561 ts_tablet_manager.cc:1403] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.081s	user 0.009s	sys 0.008s
I20260709 13:14:22.351301 11560 ts_tablet_manager.cc:1434] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:14:22.351550 11561 raft_consensus.cc:359] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.350872 11266 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.352175 11561 raft_consensus.cc:385] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.352560 11561 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.352993 11559 tablet_bootstrap.cc:654] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.353451 11561 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.355973 11561 ts_tablet_manager.cc:1434] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:22.356824 11561 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:22.361954 11561 tablet_bootstrap.cc:654] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.352231 11560 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.366385 11559 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.365890 11264 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.366850 11559 ts_tablet_manager.cc:1403] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.000s
I20260709 13:14:22.371584 11559 raft_consensus.cc:359] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.372241 11559 raft_consensus.cc:385] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.372349 11560 tablet_bootstrap.cc:654] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.372515 11559 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.373230 11559 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.375481 11559 ts_tablet_manager.cc:1434] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:14:22.376883 11559 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.380841 11566 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.381318 11566 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.382424 11613 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.382846 11613 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.383373 11566 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.384640 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.390764 11444 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.384685 11613 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.385689 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.387423 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.392330 11443 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.392401 11369 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.389343 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.393955 11596 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:22.394292 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:22.391713 11484 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.395135 11613 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.386940 11559 tablet_bootstrap.cc:654] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.395548 11613 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.388048 11592 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.395915 11613 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.396210 11592 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.397154 11368 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.399080 11561 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:22.399634 11561 ts_tablet_manager.cc:1403] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.005s
I20260709 13:14:22.400871 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.401500 11592 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.399066 11566 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.396749 11560 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.403126 11566 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.403353 11560 ts_tablet_manager.cc:1403] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.004s
I20260709 13:14:22.406570 11560 raft_consensus.cc:359] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.407232 11560 raft_consensus.cc:385] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.402340 11561 raft_consensus.cc:359] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.403625 11566 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.407601 11561 raft_consensus.cc:385] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.407929 11561 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.408687 11561 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.411434 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.412040 11443 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:22.413225 11687 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.413565 11335 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.415532 11596 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.415858 11596 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.416056 11559 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.416200 11596 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.416513 11559 ts_tablet_manager.cc:1403] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.005s
I20260709 13:14:22.418991 11559 raft_consensus.cc:359] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.419622 11559 raft_consensus.cc:385] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.419864 11559 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.420444 11559 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.422461 11559 ts_tablet_manager.cc:1434] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:14:22.423312 11559 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.426614 11560 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.427774 11561 ts_tablet_manager.cc:1434] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 13:14:22.428686 11561 tablet_bootstrap.cc:492] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:22.425577 11590 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:22.436175 11560 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.437211 11613 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.438278 11560 ts_tablet_manager.cc:1434] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.002s
I20260709 13:14:22.439169 11560 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.439179 11613 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.444224 11613 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.444893 11613 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.448822 11566 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.449373 11613 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.449585 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.450131 11367 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.450963 11566 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.459048 11614 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.459501 11614 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.460815 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.461377 11443 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.461464 11614 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.463229 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.463817 11442 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.467756 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.468421 11444 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.469910 11484 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.470893 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.471554 11519 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:22.471498 11444 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.472648 11614 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.473032 11614 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.475544 11484 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.492604 11613 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.495203 11613 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.498261 11561 tablet_bootstrap.cc:654] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.474838 11614 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.501468 11560 tablet_bootstrap.cc:654] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.495587 11613 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.507864 11710 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.519191 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.519788 11443 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.520565 11612 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.522429 11367 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.524904 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.525571 11369 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.517465 11710 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:22.526183 11483 leader_election.cc:343] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.527137 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:22.527678 11596 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.528558 11612 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.513978 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.529033 11612 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:22.529572 11368 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 1: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:14:22.529695 11596 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.529827 11612 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.531901 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:22.532580 11519 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:14:22.530777 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:22.531255 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.533440 11369 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:22.533934 11442 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.534664 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.535518 11643 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.534173 11559 tablet_bootstrap.cc:654] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.537928 11643 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.538385 11643 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.527948 11710 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.536163 11590 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:14:22.547601 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.555090 11614 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.556687 11643 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.557150 11614 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.554651 11444 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:22.558072 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.546006 11484 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:22.561679 11560 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:22.562039 11560 ts_tablet_manager.cc:1403] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.123s	user 0.013s	sys 0.002s
I20260709 13:14:22.558686 11368 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.558699 11643 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.564015 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.564316 11335 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:22.564600 11369 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.566883 11592 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.577951 11369 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:22.588076 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.589169 11561 tablet_bootstrap.cc:492] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:22.589632 11561 ts_tablet_manager.cc:1403] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.161s	user 0.011s	sys 0.007s
I20260709 13:14:22.590472 11710 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.577517 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.590939 11710 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:22.581037 11265 catalog_manager.cc:5697] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:22.592026 11561 raft_consensus.cc:359] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.593845 11561 raft_consensus.cc:385] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.594197 11561 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.561358 11596 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:14:22.591370 11518 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:22.591754 11710 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.575911 11614 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:14:22.577219 11613 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.563194 11612 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.578022 11643 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:14:22.600419 11560 raft_consensus.cc:359] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.601887 11560 raft_consensus.cc:385] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.601250 11565 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.602661 11560 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.594874 11561 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.608521 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:14:22.611574 11483 leader_election.cc:343] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.613926 11561 ts_tablet_manager.cc:1434] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:14:22.614886 11561 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:22.611493 11560 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.619084 11560 ts_tablet_manager.cc:1434] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.000s
I20260709 13:14:22.620316 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.621443 11559 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.621750 11560 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.605198 11612 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:22.624001 11559 ts_tablet_manager.cc:1403] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.201s	user 0.008s	sys 0.010s
I20260709 13:14:22.620741 11687 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.620862 11443 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.611717 11613 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.652259 11368 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.655000 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:22.655580 11518 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.657780 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:22.658811 11613 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.659245 11613 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:22.660066 11612 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.660032 11613 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.661876 11443 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.652655 11710 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.681366 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.691354 11445 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.694077 11518 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:22.713423 11559 raft_consensus.cc:359] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.713939 11559 raft_consensus.cc:385] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.714155 11559 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.708701 11265 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.680876 11710 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.718400 11745 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.719053 11745 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.726734 11559 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.752029 11368 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.756202 11445 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.761157 11560 tablet_bootstrap.cc:654] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.761088 11710 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.762260 11559 ts_tablet_manager.cc:1434] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.136s	user 0.026s	sys 0.018s
I20260709 13:14:22.763718 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.766535 11745 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.769086 11559 tablet_bootstrap.cc:492] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:22.769701 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.770294 11366 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
W20260709 13:14:22.771291 11483 leader_election.cc:343] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.771389 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.774590 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:22.775590 11710 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.775969 11710 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.776301 11710 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.777573 11643 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:14:22.779114 11408 leader_election.cc:343] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.788913 11565 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.789461 11565 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.790143 11264 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:22.792706 11752 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.105s	user 0.003s	sys 0.000s
I20260709 13:14:22.795379 11565 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.796217 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.797591 11559 tablet_bootstrap.cc:654] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.795126 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:14:22.797889 11409 leader_election.cc:343] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.787055 11710 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.798316 11445 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:22.798537 11592 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:22.800529 11710 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.801949 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.802558 11366 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.809294 11561 tablet_bootstrap.cc:654] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.815172 11484 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:22.822077 11745 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.822708 11745 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:22.830467 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:22.831471 11614 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:22.831840 11614 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.832165 11614 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.832383 11745 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.825469 11710 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.843808 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:22.844066 11710 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.844955 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.846526 11710 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:14:22.849584 11408 leader_election.cc:343] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.850432 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.847296 11614 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.852756 11267 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:22.848320 11710 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.855980 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:22.849305 11366 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:22.865491 11516 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:14:22.863464 11408 leader_election.cc:343] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.868980 11614 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.870450 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.871212 11565 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:22.871783 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:22.879045 11559 tablet_bootstrap.cc:492] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:22.879575 11559 ts_tablet_manager.cc:1403] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.111s	user 0.025s	sys 0.008s
I20260709 13:14:22.883786 11596 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.889797 11559 raft_consensus.cc:359] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.891454 11559 raft_consensus.cc:385] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.893124 11559 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.857936 11710 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.897168 11264 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:22.901273 11560 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
W20260709 13:14:22.892539 11409 leader_election.cc:343] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.900619 11559 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.902134 11560 ts_tablet_manager.cc:1403] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.281s	user 0.019s	sys 0.002s
I20260709 13:14:22.900173 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:22.909467 11559 ts_tablet_manager.cc:1434] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:14:22.912304 11559 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
W20260709 13:14:22.919991 11483 leader_election.cc:343] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.917075 11710 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:22.921046 11560 raft_consensus.cc:359] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.922331 11560 raft_consensus.cc:385] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.922623 11560 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.923674 11560 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.928159 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.929204 11565 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.929500 11560 ts_tablet_manager.cc:1434] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:14:22.932708 11560 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:22.935649 11614 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.940536 11565 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.941851 11710 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.942050 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:22.943987 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 79609.9us
I20260709 13:14:22.944831 11561 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:22.945257 11561 ts_tablet_manager.cc:1403] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.331s	user 0.019s	sys 0.007s
W20260709 13:14:22.947059 11408 leader_election.cc:343] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.947364 11444 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:22.949867 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.950039 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.951161 11641 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:14:22.951326 11409 leader_election.cc:343] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:22.951669 11641 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:22.951828 11409 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:22.952548 11641 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.956462 11643 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:22.960611 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:22.961604 11561 raft_consensus.cc:359] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.965898 11561 raft_consensus.cc:385] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:22.966187 11561 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:22.967321 11614 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:22.968135 11614 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:22.969437 11559 tablet_bootstrap.cc:654] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.969079 11614 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.967676 11561 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.972509 11758 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.973489 11758 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:22.975872 11758 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:22.978492 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:22.980441 11366 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:22.982126 11745 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:22.988440 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:22.988545 11267 catalog_manager.cc:5697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:22.990864 11484 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:22.993161 11560 tablet_bootstrap.cc:654] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:22.996549 11561 ts_tablet_manager.cc:1434] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.002s
I20260709 13:14:22.997560 11561 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:22.999004 11745 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.038480 11561 tablet_bootstrap.cc:654] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.050813 11745 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:23.050396 11264 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.051743 11687 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.054133 11687 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.056509 11687 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:14:23.052592 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 61822.2us
I20260709 13:14:23.059108 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.062198 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.062939 11445 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:23.064637 11333 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:23.065959 11687 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.067764 11768 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:23.070063 11614 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.075613 11768 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.081080 11561 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:23.082149 11561 ts_tablet_manager.cc:1403] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.005s
I20260709 13:14:23.084908 11561 raft_consensus.cc:359] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.089728 11560 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.092165 11560 ts_tablet_manager.cc:1403] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.160s	user 0.015s	sys 0.041s
I20260709 13:14:23.085317 11559 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:23.093237 11559 ts_tablet_manager.cc:1403] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.181s	user 0.005s	sys 0.011s
I20260709 13:14:23.091894 11758 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.095130 11560 raft_consensus.cc:359] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.096505 11560 raft_consensus.cc:385] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.096859 11560 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.097555 11560 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.099550 11560 ts_tablet_manager.cc:1434] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:23.088114 11561 raft_consensus.cc:385] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.099946 11561 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.100690 11561 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.102782 11561 ts_tablet_manager.cc:1434] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:14:23.103701 11561 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:23.104578 11560 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.112614 11614 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.113080 11559 raft_consensus.cc:359] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.113838 11559 raft_consensus.cc:385] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.114121 11559 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.114755 11559 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.116513 11560 tablet_bootstrap.cc:654] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.117697 11561 tablet_bootstrap.cc:654] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.112664 11687 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:23.123899 11758 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:23.126470 11559 ts_tablet_manager.cc:1434] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:14:23.138682 11559 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:23.139135 11560 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.139647 11560 ts_tablet_manager.cc:1403] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.005s
I20260709 13:14:23.142180 11560 raft_consensus.cc:359] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.144721 11561 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:23.145321 11561 ts_tablet_manager.cc:1403] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.004s
I20260709 13:14:23.147372 11560 raft_consensus.cc:385] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.147732 11560 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.151059 11561 raft_consensus.cc:359] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.151788 11561 raft_consensus.cc:385] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.152041 11561 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.152681 11561 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.154721 11561 ts_tablet_manager.cc:1434] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:14:23.155705 11561 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:23.161352 11561 tablet_bootstrap.cc:654] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.163803 11560 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.166064 11560 ts_tablet_manager.cc:1434] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.026s	user 0.001s	sys 0.004s
I20260709 13:14:23.171294 11560 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.174737 11559 tablet_bootstrap.cc:654] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.176847 11560 tablet_bootstrap.cc:654] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.189364 11560 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.189828 11560 ts_tablet_manager.cc:1403] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.004s
I20260709 13:14:23.192309 11560 raft_consensus.cc:359] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.192950 11560 raft_consensus.cc:385] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.193205 11560 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.193816 11560 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.200655 11561 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:23.201135 11561 ts_tablet_manager.cc:1403] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.046s	user 0.005s	sys 0.013s
I20260709 13:14:23.201877 11559 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:23.202280 11559 ts_tablet_manager.cc:1403] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.004s
I20260709 13:14:23.203177 11561 raft_consensus.cc:359] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.203771 11561 raft_consensus.cc:385] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.203954 11560 ts_tablet_manager.cc:1434] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:14:23.204041 11561 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.204275 11559 raft_consensus.cc:359] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.204888 11560 tablet_bootstrap.cc:492] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.204712 11561 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.206734 11561 ts_tablet_manager.cc:1434] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:14:23.208112 11561 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:23.210435 11560 tablet_bootstrap.cc:654] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.213551 11559 raft_consensus.cc:385] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.213729 11561 tablet_bootstrap.cc:654] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.213883 11559 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.214684 11559 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.226208 11559 ts_tablet_manager.cc:1434] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:14:23.234822 11559 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:23.236605 11561 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:23.237182 11561 ts_tablet_manager.cc:1403] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.007s
I20260709 13:14:23.237432 11758 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.237946 11758 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.240095 11758 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:23.240906 11559 tablet_bootstrap.cc:654] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.240054 11561 raft_consensus.cc:359] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.241278 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.241261 11561 raft_consensus.cc:385] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.242224 11561 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.242451 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.243038 11444 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:23.242897 11561 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.248775 11560 tablet_bootstrap.cc:492] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.252736 11559 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:23.256927 11560 ts_tablet_manager.cc:1403] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.052s	user 0.024s	sys 0.023s
I20260709 13:14:23.257005 11559 ts_tablet_manager.cc:1403] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 13:14:23.244313 11368 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:23.259487 11560 raft_consensus.cc:359] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.260155 11560 raft_consensus.cc:385] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.260432 11560 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.244263 11484 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.261081 11560 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.261499 11758 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:23.261667 11612 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.261909 11758 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.262162 11612 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.263139 11560 ts_tablet_manager.cc:1434] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:14:23.263347 11561 ts_tablet_manager.cc:1434] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:14:23.259744 11559 raft_consensus.cc:359] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.264158 11612 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:23.264614 11561 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:23.266017 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.266615 11366 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:23.267876 11560 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.270159 11561 tablet_bootstrap.cc:654] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.271461 11559 raft_consensus.cc:385] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.271773 11559 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.272426 11559 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.273247 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:23.273857 11560 tablet_bootstrap.cc:654] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.275672 11614 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:23.276033 11614 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.276373 11614 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.276250 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.277311 11444 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:23.280455 11560 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.280927 11560 ts_tablet_manager.cc:1403] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:14:23.282090 11758 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.282315 11565 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.282771 11565 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.283401 11560 raft_consensus.cc:359] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.284019 11560 raft_consensus.cc:385] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.284281 11560 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.284762 11565 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.284834 11560 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.286211 11565 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.286412 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.286676 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.286645 11565 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:23.288101 11409 leader_election.cc:343] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.290355 11614 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.292269 11614 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:23.292946 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:23.293579 11368 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:23.294471 11408 leader_election.cc:343] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.294687 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:23.294919 11408 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:23.295275 11444 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.295480 11559 ts_tablet_manager.cc:1434] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.002s
I20260709 13:14:23.296486 11559 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:23.300186 11641 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:23.289361 11565 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.289525 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.302480 11366 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:23.303680 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.304353 11560 ts_tablet_manager.cc:1434] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 13:14:23.289922 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.305004 11516 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:23.306576 11565 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:23.306988 11565 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.307309 11565 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.310834 11368 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:23.312006 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:23.312203 11612 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.312911 11745 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:23.313383 11745 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:23.314185 11745 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.315357 11560 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.315070 11565 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.321664 11565 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.322475 11560 tablet_bootstrap.cc:654] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.323268 11559 tablet_bootstrap.cc:654] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.323414 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:23.323997 11516 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.289228 11758 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.330482 11758 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:23.333173 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:23.333722 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:23.334321 11442 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.333842 11366 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.337399 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:23.338898 11368 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.341511 11442 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:23.335850 11516 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:23.342880 11408 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.342980 11484 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.343931 11366 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:23.344242 11641 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:23.344697 11641 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:23.345650 11612 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:23.345557 11641 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.346349 11612 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:23.347227 11612 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.349550 11559 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:23.350019 11559 ts_tablet_manager.cc:1403] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.000s
I20260709 13:14:23.356174 11559 raft_consensus.cc:359] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.356897 11559 raft_consensus.cc:385] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.357159 11559 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.360121 11559 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.362423 11559 ts_tablet_manager.cc:1434] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:14:23.363457 11559 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:23.366605 11368 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
W20260709 13:14:23.368511 11483 consensus_peers.cc:597] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:23.371374 11561 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:23.371934 11561 ts_tablet_manager.cc:1403] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.108s	user 0.010s	sys 0.014s
I20260709 13:14:23.372363 11444 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:23.373952 11561 raft_consensus.cc:359] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.374454 11561 raft_consensus.cc:385] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.374728 11561 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.375372 11561 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.376304 11560 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.372012 11264 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.376740 11560 ts_tablet_manager.cc:1403] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.021s
I20260709 13:14:23.379794 11560 raft_consensus.cc:359] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.381071 11267 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:23.385390 11561 ts_tablet_manager.cc:1434] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 13:14:23.380427 11560 raft_consensus.cc:385] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.386010 11560 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.386343 11561 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:23.386850 11559 tablet_bootstrap.cc:654] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.386647 11560 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.388954 11560 ts_tablet_manager.cc:1434] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:14:23.391942 11561 tablet_bootstrap.cc:654] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.398154 11559 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:23.399458 11560 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.398607 11559 ts_tablet_manager.cc:1403] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.003s
I20260709 13:14:23.403200 11265 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.405254 11560 tablet_bootstrap.cc:654] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.407651 11559 raft_consensus.cc:359] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.408118 11641 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.408365 11559 raft_consensus.cc:385] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.408725 11559 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.408631 11641 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.409426 11559 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.410848 11641 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.411247 11559 ts_tablet_manager.cc:1434] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:14:23.412150 11559 tablet_bootstrap.cc:492] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:23.412194 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.413141 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:23.414624 11409 leader_election.cc:343] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:23.415548 11408 leader_election.cc:343] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.415954 11408 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:23.416942 11768 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:23.417287 11612 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.418151 11559 tablet_bootstrap.cc:654] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.421845 11687 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.422323 11687 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.424345 11687 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.427001 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.427647 11560 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.427688 11444 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:23.428221 11560 ts_tablet_manager.cc:1403] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.002s
I20260709 13:14:23.429003 11641 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.429450 11641 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.430289 11335 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.431653 11596 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:23.432236 11596 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.432536 11596 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.441361 11768 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.435603 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.443135 11516 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:23.470504 11596 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.481554 11641 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.506037 11559 tablet_bootstrap.cc:492] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:23.506121 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:23.506662 11559 ts_tablet_manager.cc:1403] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.095s	user 0.028s	sys 0.026s
I20260709 13:14:23.506803 11444 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:23.509344 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 43810.6us
I20260709 13:14:23.510011 11560 raft_consensus.cc:359] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.510700 11560 raft_consensus.cc:385] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.510979 11560 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.518172 11444 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:23.517956 11612 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.519164 11335 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.520923 11560 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.526777 11641 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.531019 11559 raft_consensus.cc:359] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.531771 11559 raft_consensus.cc:385] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.532037 11559 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.532105 11561 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:23.532622 11561 ts_tablet_manager.cc:1403] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.146s	user 0.010s	sys 0.011s
I20260709 13:14:23.533386 11560 ts_tablet_manager.cc:1434] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.000s
I20260709 13:14:23.537540 11560 tablet_bootstrap.cc:492] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.530750 11641 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.548243 11559 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.553231 11808 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:23.553784 11808 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:23.554625 11808 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.559123 11559 ts_tablet_manager.cc:1434] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.052s	user 0.004s	sys 0.004s
I20260709 13:14:23.549304 11596 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.574343 11560 tablet_bootstrap.cc:654] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.579296 11758 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.575760 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:23.562942 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 56996.5us
I20260709 13:14:23.583690 11561 raft_consensus.cc:359] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.585515 11561 raft_consensus.cc:385] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.587240 11561 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.587965 11561 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.579799 11758 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.590070 11768 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.598404 11565 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.593322 11517 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:23.600980 11565 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.591706 11768 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.605762 11561 ts_tablet_manager.cc:1434] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 13:14:23.606861 11561 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:23.609894 11641 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.616423 11758 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:23.616696 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:23.617365 11518 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.707022 11768 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.642925 11559 tablet_bootstrap.cc:492] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:23.706892 11596 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.681188 11565 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.720643 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.717814 11745 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.724059 11596 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.736522 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.742867 11370 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:23.744644 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.747238 11560 tablet_bootstrap.cc:492] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.747757 11560 ts_tablet_manager.cc:1403] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.214s	user 0.011s	sys 0.023s
I20260709 13:14:23.748327 11561 tablet_bootstrap.cc:654] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.731181 11408 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.747726 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.732659 11745 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.750221 11369 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:23.768675 11565 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:23.769119 11565 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.769448 11565 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.772908 11768 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.784242 11560 raft_consensus.cc:359] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.786405 11560 raft_consensus.cc:385] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.786980 11560 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.777724 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.791589 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.794162 11518 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:23.794819 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.796185 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.798849 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.796465 11596 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.801221 11560 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.810591 11560 ts_tablet_manager.cc:1434] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.004s
I20260709 13:14:23.812119 11560 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:23.803006 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:23.826566 11267 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:23.830209 11408 leader_election.cc:343] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.833715 11745 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:23.834023 11641 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:23.837363 11560 tablet_bootstrap.cc:654] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.849596 11641 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.854414 11641 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.838040 11565 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:23.839001 11408 leader_election.cc:343] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.840579 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:23.867244 11408 leader_election.cc:343] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.865731 11519 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:23.871820 11768 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.872314 11768 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.873390 11613 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.873838 11613 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.874253 11768 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.875746 11613 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:23.867184 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.880889 11559 tablet_bootstrap.cc:654] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.882633 11561 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:23.883028 11561 ts_tablet_manager.cc:1403] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.276s	user 0.014s	sys 0.020s
I20260709 13:14:23.882119 11442 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:14:23.883621 11409 leader_election.cc:343] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.888008 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.893263 11641 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.894078 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:23.899060 11809 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:23.899426 11809 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.899765 11809 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.901139 11641 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.901017 11561 raft_consensus.cc:359] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.901758 11561 raft_consensus.cc:385] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.902029 11561 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
W20260709 13:14:23.902721 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:23.902678 11561 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.910159 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.910629 11565 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.891757 11409 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:23.914343 11811 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:23.916052 11409 leader_election.cc:343] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.917104 11745 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:14:23.922245 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:23.919180 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:23.924710 11565 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:23.925961 11561 ts_tablet_manager.cc:1434] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 13:14:23.928741 11560 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:23.929188 11561 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:23.929548 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:23.929821 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.927328 11641 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 13:14:23.930862 11483 leader_election.cc:343] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.931599 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:23.929204 11560 ts_tablet_manager.cc:1403] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.117s	user 0.010s	sys 0.016s
W20260709 13:14:23.936992 11483 leader_election.cc:343] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.936885 11613 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:23.916544 11809 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.940364 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:23.941032 11444 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:23.944139 11809 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:23.945686 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:23.946319 11367 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.953604 11367 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:23.954165 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:23.954782 11519 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:23.956291 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:14:23.957731 11408 leader_election.cc:343] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:23.963485 11408 leader_election.cc:343] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.966780 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:23.969398 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:23.976104 11367 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 1: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:14:23.979388 11409 leader_election.cc:343] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.981007 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:23.982338 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:23.985425 11565 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:23.986124 11565 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:23.986537 11565 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:14:23.987147 11484 leader_election.cc:343] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:23.988273 11519 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:23.990274 11561 tablet_bootstrap.cc:654] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:23.995114 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:23.995883 11444 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:23.996275 11560 raft_consensus.cc:359] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.997020 11560 raft_consensus.cc:385] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:23.997273 11560 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:23.998039 11560 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:23.987890 11484 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:24.001348 11560 ts_tablet_manager.cc:1434] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.000s
I20260709 13:14:23.994138 11614 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.002290 11560 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.002722 11613 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.002992 11612 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:23.987347 11565 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.007158 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:24.029067 11559 tablet_bootstrap.cc:492] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.029464 11559 ts_tablet_manager.cc:1403] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.387s	user 0.024s	sys 0.012s
I20260709 13:14:24.029352 11614 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.029829 11614 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.030171 11614 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.035768 11363 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.032109 11559 raft_consensus.cc:359] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.040761 11559 raft_consensus.cc:385] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.041062 11559 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.041662 11559 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.061157 11559 ts_tablet_manager.cc:1434] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.002s
I20260709 13:14:24.062204 11559 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
W20260709 13:14:24.075130 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 92879.5us
I20260709 13:14:24.084408 11335 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:24.105024 11408 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:24.105683 11264 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.093380 11809 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.108433 11561 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.108863 11561 ts_tablet_manager.cc:1403] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.180s	user 0.026s	sys 0.009s
I20260709 13:14:24.112370 11561 raft_consensus.cc:359] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.113282 11561 raft_consensus.cc:385] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.113643 11561 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.114260 11561 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.115710 11561 ts_tablet_manager.cc:1434] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:24.115887 11614 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.117847 11614 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.120398 11560 tablet_bootstrap.cc:654] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.120919 11561 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.119910 11826 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.132001 11825 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:24.121913 11745 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:14:24.136804 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:24.137878 11826 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:14:24.150322 11483 leader_election.cc:343] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.150146 11826 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.154083 11809 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:24.155746 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:24.156363 11440 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.161885 11813 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:24.163343 11559 tablet_bootstrap.cc:654] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.225417 11560 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.225867 11560 ts_tablet_manager.cc:1403] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.224s	user 0.009s	sys 0.020s
I20260709 13:14:24.234860 11559 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.241510 11560 raft_consensus.cc:359] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.247226 11559 ts_tablet_manager.cc:1403] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.185s	user 0.011s	sys 0.016s
I20260709 13:14:24.247332 11560 raft_consensus.cc:385] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.251694 11560 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.252908 11440 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:24.256036 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:24.263989 11613 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.264505 11613 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:24.265232 11613 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.271656 11559 raft_consensus.cc:359] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.272297 11559 raft_consensus.cc:385] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.272538 11559 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.273239 11561 tablet_bootstrap.cc:654] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.273111 11559 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.261286 11560 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.275090 11559 ts_tablet_manager.cc:1434] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.000s
I20260709 13:14:24.275974 11559 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:24.274662 11267 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.276247 11560 ts_tablet_manager.cc:1434] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.003s
I20260709 13:14:24.277086 11560 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.280689 11614 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.281181 11614 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.284552 11614 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.284979 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.287609 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.288219 11445 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
W20260709 13:14:24.288340 11483 leader_election.cc:343] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.290386 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:24.292012 11614 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.292366 11614 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.292704 11614 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.296603 11559 tablet_bootstrap.cc:654] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.304507 11826 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.308640 11265 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.302559 11614 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.309391 11826 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.312281 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:24.313313 11614 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.313378 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.313551 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:24.312939 11440 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:24.314919 11409 leader_election.cc:343] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:24.319561 11483 leader_election.cc:343] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.322309 11561 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.322786 11561 ts_tablet_manager.cc:1403] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.202s	user 0.007s	sys 0.012s
I20260709 13:14:24.317178 11560 tablet_bootstrap.cc:654] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.324651 11826 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.324829 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:24.326131 11408 leader_election.cc:343] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.326611 11408 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:24.327430 11826 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.328857 11561 raft_consensus.cc:359] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.329593 11561 raft_consensus.cc:385] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.329859 11561 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.330521 11561 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.333616 11440 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:24.335527 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:24.335973 11561 ts_tablet_manager.cc:1434] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.005s
I20260709 13:14:24.336679 11612 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.337213 11612 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:24.337893 11559 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.338495 11559 ts_tablet_manager.cc:1403] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.014s
I20260709 13:14:24.337961 11612 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.341668 11559 raft_consensus.cc:359] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.342417 11559 raft_consensus.cc:385] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.342713 11559 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.344995 11561 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.351684 11559 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.353492 11265 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.354876 11561 tablet_bootstrap.cc:654] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.354209 11559 ts_tablet_manager.cc:1434] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:14:24.358717 11559 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:24.364073 11559 tablet_bootstrap.cc:654] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.369390 11612 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.369942 11612 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.375150 11559 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.375639 11559 ts_tablet_manager.cc:1403] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:14:24.376506 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.376662 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.377194 11440 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:24.378244 11612 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.378060 11559 raft_consensus.cc:359] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.378297 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:24.379213 11613 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.379590 11613 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:24.379978 11483 leader_election.cc:343] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.379925 11613 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.384991 11560 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.385461 11560 ts_tablet_manager.cc:1403] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.109s	user 0.009s	sys 0.021s
I20260709 13:14:24.386732 11561 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.387140 11561 ts_tablet_manager.cc:1403] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.021s
I20260709 13:14:24.387465 11613 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.389580 11561 raft_consensus.cc:359] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.390225 11561 raft_consensus.cc:385] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.390475 11561 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.391062 11561 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.392863 11613 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.393117 11561 ts_tablet_manager.cc:1434] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:24.393702 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:24.387913 11560 raft_consensus.cc:359] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.394284 11560 raft_consensus.cc:385] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.394543 11560 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.394721 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:24.395262 11440 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:24.395390 11483 leader_election.cc:343] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.394740 11561 tablet_bootstrap.cc:492] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.396201 11560 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.398895 11560 ts_tablet_manager.cc:1434] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:14:24.388087 11559 raft_consensus.cc:385] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.399541 11559 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.400182 11559 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.401659 11560 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.402550 11440 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:24.403831 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:24.406104 11559 ts_tablet_manager.cc:1434] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:14:24.406597 11561 tablet_bootstrap.cc:654] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.406774 11614 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.407207 11614 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:24.407958 11614 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.408639 11559 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:24.417287 11560 tablet_bootstrap.cc:654] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.418979 11559 tablet_bootstrap.cc:654] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.432365 11614 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.432845 11614 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.436709 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.438284 11614 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.442986 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.443933 11367 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:24.445362 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:24.446900 11614 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.442412 11265 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.452759 11826 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:24.459772 11614 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.462739 11809 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.463222 11809 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.465273 11809 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.466775 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:24.467365 11517 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:24.468564 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:24.469363 11809 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.469745 11809 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.470112 11809 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.474237 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.474853 11440 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:24.479985 11561 tablet_bootstrap.cc:492] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.480484 11561 ts_tablet_manager.cc:1403] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.025s
I20260709 13:14:24.483139 11561 raft_consensus.cc:359] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.483865 11561 raft_consensus.cc:385] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.484120 11561 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.484022 11826 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:24.484926 11561 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.486410 11561 ts_tablet_manager.cc:1434] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:24.487294 11561 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.488754 11560 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.489249 11560 ts_tablet_manager.cc:1403] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.088s	user 0.012s	sys 0.017s
I20260709 13:14:24.492095 11560 raft_consensus.cc:359] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.492779 11560 raft_consensus.cc:385] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.493024 11560 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.493534 11561 tablet_bootstrap.cc:654] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.493645 11560 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.498783 11559 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.499295 11559 ts_tablet_manager.cc:1403] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.026s
I20260709 13:14:24.502166 11559 raft_consensus.cc:359] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.502871 11559 raft_consensus.cc:385] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.503129 11559 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.504797 11560 ts_tablet_manager.cc:1434] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.005s
I20260709 13:14:24.505860 11560 tablet_bootstrap.cc:492] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.511925 11559 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.514551 11559 ts_tablet_manager.cc:1434] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:14:24.515590 11559 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:24.518878 11560 tablet_bootstrap.cc:654] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.524677 11559 tablet_bootstrap.cc:654] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.521755 11809 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.531919 11561 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.532374 11561 ts_tablet_manager.cc:1403] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.026s
I20260709 13:14:24.532964 11809 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.534837 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:24.535426 11519 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.537410 11561 raft_consensus.cc:359] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.538132 11561 raft_consensus.cc:385] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.538405 11561 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.539740 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:24.540251 11440 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.539556 11561 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.543445 11561 ts_tablet_manager.cc:1434] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:14:24.545265 11560 tablet_bootstrap.cc:492] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.546900 11560 ts_tablet_manager.cc:1403] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.006s
I20260709 13:14:24.546025 11559 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.547472 11559 ts_tablet_manager.cc:1403] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.004s
I20260709 13:14:24.546932 11561 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.549633 11560 raft_consensus.cc:359] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.550287 11560 raft_consensus.cc:385] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.550539 11560 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.550586 11519 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:24.552789 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:24.553563 11809 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.554032 11809 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:24.554824 11809 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.558085 11440 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:24.554032 11561 tablet_bootstrap.cc:654] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.561177 11559 raft_consensus.cc:359] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.561843 11559 raft_consensus.cc:385] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.562091 11559 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.551216 11560 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.562745 11559 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.564746 11559 ts_tablet_manager.cc:1434] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:14:24.565582 11559 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:24.568071 11560 ts_tablet_manager.cc:1434] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:14:24.568544 11809 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.568990 11560 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.569016 11809 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.569763 11265 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.570919 11559 tablet_bootstrap.cc:654] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.571374 11809 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.573086 11805 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.576687 11560 tablet_bootstrap.cc:654] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.573521 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:24.576875 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.577492 11440 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:24.576335 11805 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:24.577910 11333 leader_election.cc:343] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.579751 11805 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.580293 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.580958 11445 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:24.583328 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:24.584993 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:24.587960 11561 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.588443 11561 ts_tablet_manager.cc:1403] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.042s	user 0.003s	sys 0.008s
I20260709 13:14:24.590905 11561 raft_consensus.cc:359] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.591892 11561 raft_consensus.cc:385] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.592198 11561 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.592439 11809 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.592901 11561 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.594274 11335 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:24.595079 11561 ts_tablet_manager.cc:1434] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.002s
I20260709 13:14:24.594986 11813 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.595571 11813 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.595932 11813 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.596287 11561 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.600795 11809 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.602996 11813 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.606407 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:24.607018 11440 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.612111 11561 tablet_bootstrap.cc:654] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.613735 11440 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:24.615113 11335 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:24.616091 11805 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.616529 11805 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:24.617308 11805 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.618152 11559 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.618649 11559 ts_tablet_manager.cc:1403] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.013s
I20260709 13:14:24.620150 11641 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.620646 11641 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.621515 11559 raft_consensus.cc:359] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.622208 11559 raft_consensus.cc:385] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.622713 11813 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.623943 11560 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.624601 11560 ts_tablet_manager.cc:1403] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.056s	user 0.020s	sys 0.026s
I20260709 13:14:24.624382 11559 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.625772 11559 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.624341 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.627723 11560 raft_consensus.cc:359] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.628376 11560 raft_consensus.cc:385] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.628654 11560 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
W20260709 13:14:24.628854 11409 leader_election.cc:343] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.629251 11560 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.630398 11559 ts_tablet_manager.cc:1434] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:14:24.631201 11560 ts_tablet_manager.cc:1434] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:14:24.632786 11561 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.635999 11561 ts_tablet_manager.cc:1403] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.040s	user 0.002s	sys 0.010s
I20260709 13:14:24.636302 11560 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.637192 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:24.638578 11408 leader_election.cc:343] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.638770 11561 raft_consensus.cc:359] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.639025 11408 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:24.639639 11561 raft_consensus.cc:385] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.640043 11561 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.640345 11565 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.640810 11561 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.633471 11559 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:24.643060 11560 tablet_bootstrap.cc:654] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.642971 11641 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.648872 11561 ts_tablet_manager.cc:1434] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:14:24.649835 11561 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.655246 11612 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.656216 11559 tablet_bootstrap.cc:654] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.655768 11612 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.655486 11561 tablet_bootstrap.cc:654] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.660763 11612 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.660867 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.661613 11367 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:24.666062 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:24.667804 11483 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:24.669189 11612 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.669566 11612 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.669898 11612 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:24.671379 11333 leader_election.cc:343] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.673020 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.675745 11440 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:24.679836 11560 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.680227 11560 ts_tablet_manager.cc:1403] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.044s	user 0.026s	sys 0.017s
I20260709 13:14:24.680882 11267 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.682406 11560 raft_consensus.cc:359] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.682883 11560 raft_consensus.cc:385] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.683075 11560 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.683676 11560 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.685611 11560 ts_tablet_manager.cc:1434] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:24.685760 11813 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.686436 11560 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.686183 11813 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.688097 11813 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.689273 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.689865 11445 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:24.690565 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:24.691090 11519 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:24.690934 11745 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:24.692170 11560 tablet_bootstrap.cc:654] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.693346 11335 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:24.694209 11805 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.694551 11805 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.694872 11805 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.701833 11641 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.702032 11612 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.702422 11641 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.704499 11641 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.705752 11560 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.706190 11560 ts_tablet_manager.cc:1403] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.006s
I20260709 13:14:24.705936 11805 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.707858 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.708103 11805 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.709389 11560 raft_consensus.cc:359] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.709775 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:24.710060 11560 raft_consensus.cc:385] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.710481 11560 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.711112 11560 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.711504 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:24.712013 11440 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.713171 11560 ts_tablet_manager.cc:1434] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:24.714104 11560 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
W20260709 13:14:24.716742 11409 leader_election.cc:343] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.710482 11517 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.720203 11560 tablet_bootstrap.cc:654] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.720280 11559 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:24.720844 11559 ts_tablet_manager.cc:1403] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.088s	user 0.012s	sys 0.019s
I20260709 13:14:24.723505 11559 raft_consensus.cc:359] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.724170 11559 raft_consensus.cc:385] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.724735 11559 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.706065 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:24.728156 11559 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.738512 11559 ts_tablet_manager.cc:1434] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:14:24.741909 11559 tablet_bootstrap.cc:492] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
W20260709 13:14:24.736945 11408 leader_election.cc:343] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.742573 11408 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:24.748458 11565 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.748597 11440 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:24.752008 11517 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:24.754376 11333 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:24.756235 11808 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.765441 11565 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.765690 11808 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:24.766557 11808 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.775588 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:24.781095 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:24.798290 11445 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.802508 11561 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:24.803052 11561 ts_tablet_manager.cc:1403] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.153s	user 0.035s	sys 0.021s
I20260709 13:14:24.778162 11368 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.787465 11612 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.819756 11614 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.820955 11560 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:24.775504 11565 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.825894 11560 ts_tablet_manager.cc:1403] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.006s
I20260709 13:14:24.825739 11614 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.827595 11614 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.833140 11445 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:24.781275 11826 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20260709 13:14:24.847198 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:24.850067 11614 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:24.851995 11559 tablet_bootstrap.cc:654] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:24.855795 11565 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.847854 11803 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.852931 11745 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:24.857332 11805 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.866140 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.876062 11803 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.882135 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:24.881323 11745 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:14:24.902273 11408 leader_election.cc:343] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.902879 11561 raft_consensus.cc:359] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.903669 11561 raft_consensus.cc:385] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.904440 11561 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.882862 11873 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:24.892169 11440 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:24.907989 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:24.908535 11614 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.893510 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.880513 11805 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.908880 11614 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.909663 11560 raft_consensus.cc:359] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.912277 11614 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.912357 11560 raft_consensus.cc:385] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:24.912683 11560 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:24.913296 11560 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.913869 11264 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:24.914903 11368 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:24.911813 11805 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.916145 11745 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.918370 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.918979 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:24.920835 11335 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:24.921563 11808 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:24.921901 11808 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.922206 11808 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:24.923786 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:24.925287 11441 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:24.911933 11803 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:24.930222 11560 ts_tablet_manager.cc:1434] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.104s	user 0.003s	sys 0.004s
I20260709 13:14:24.908738 11561 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.931316 11560 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:24.931865 11561 ts_tablet_manager.cc:1434] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.129s	user 0.016s	sys 0.006s
I20260709 13:14:24.932426 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.932848 11561 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:24.937395 11614 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.938730 11614 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:24.940557 11826 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.941936 11826 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:24.948881 11825 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:24.950537 11409 leader_election.cc:343] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:24.950975 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:24.951330 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:24.952201 11825 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:24.997227 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:24.965850 11808 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.017004 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:25.021703 11517 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:25.007441 11872 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.017179 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:25.030392 11443 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:25.008241 11613 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:14:25.028424 11826 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.036763 11559 tablet_bootstrap.cc:492] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:25.037256 11559 ts_tablet_manager.cc:1403] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.296s	user 0.017s	sys 0.018s
I20260709 13:14:25.039317 11559 raft_consensus.cc:359] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.040439 11559 raft_consensus.cc:385] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:25.040699 11559 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:25.041342 11559 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.043692 11559 ts_tablet_manager.cc:1434] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.005s
I20260709 13:14:25.044540 11559 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:25.062060 11561 tablet_bootstrap.cc:654] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:25.029553 11872 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.079145 11872 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.097054 11264 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:25.082500 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 68960us
I20260709 13:14:25.089248 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:25.100924 11333 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:25.105567 11560 tablet_bootstrap.cc:654] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:25.105748 11443 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:25.094937 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:25.086669 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:25.087764 11808 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:14:25.110872 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 105102us
W20260709 13:14:25.091033 11483 leader_election.cc:343] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.118322 11877 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:25.121608 11561 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:25.113140 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:25.122506 11561 ts_tablet_manager.cc:1403] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.190s	user 0.008s	sys 0.012s
I20260709 13:14:25.125097 11440 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:25.131839 11826 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.128379 11483 consensus_peers.cc:597] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:25.132303 11826 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.132871 11559 tablet_bootstrap.cc:654] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:25.122937 11813 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 13:14:25.134023 11483 leader_election.cc:343] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.134286 11826 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.142961 11877 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:25.153673 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:25.158730 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:25.166610 11612 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:25.172842 11612 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:25.173666 11612 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.174006 11873 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.176116 11873 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.180840 11335 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:25.205087 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:25.183249 11877 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:25.251365 11519 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:25.217347 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:25.252033 11873 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:25.238173 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:25.186872 11559 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:25.187726 11561 raft_consensus.cc:359] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.254993 11561 raft_consensus.cc:385] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:25.255326 11561 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:25.254774 11559 ts_tablet_manager.cc:1403] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.210s	user 0.014s	sys 0.020s
I20260709 13:14:25.267782 11519 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
W20260709 13:14:25.214332 11879 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 13:14:25.268590 11561 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:25.188153 11408 leader_election.cc:343] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.245355 11513 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:25.272483 11891 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:14:25.269295 11877 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:25.274816 11560 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:25.275276 11560 ts_tablet_manager.cc:1403] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.344s	user 0.029s	sys 0.025s
I20260709 13:14:25.281131 11560 raft_consensus.cc:359] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.281785 11560 raft_consensus.cc:385] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:25.282015 11560 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:25.283365 11877 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:25.272807 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 84689.9us
I20260709 13:14:25.361066 11613 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.372611 11809 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.373154 11613 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.373461 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:25.361089 11559 raft_consensus.cc:359] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.363875 11560 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:25.346697 11409 leader_election.cc:343] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.378872 11613 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:14:25.380187 11408 leader_election.cc:343] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.380925 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:25.382602 11641 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:25.386974 11895 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:14:25.376210 11559 raft_consensus.cc:385] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:25.382539 11560 ts_tablet_manager.cc:1434] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.107s	user 0.004s	sys 0.002s
I20260709 13:14:25.391050 11559 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:25.406695 11809 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.409467 11809 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.418902 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:25.420926 11559 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.422025 11560 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:25.425381 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:25.427958 11444 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 1: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:14:25.422552 11367 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
W20260709 13:14:25.446471 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 70701.9us
I20260709 13:14:25.439230 11559 ts_tablet_manager.cc:1434] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.171s	user 0.004s	sys 0.006s
I20260709 13:14:25.449602 11809 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:25.450949 11898 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:14:25.463165 11409 leader_election.cc:343] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:25.464160 11408 leader_election.cc:343] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.464340 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:25.471762 11884 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:25.479955 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:25.468981 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:25.502319 11825 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:25.505589 11825 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.506083 11825 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:25.509618 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 128677us
I20260709 13:14:25.510712 11561 ts_tablet_manager.cc:1434] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.347s	user 0.021s	sys 0.003s
I20260709 13:14:25.521268 11559 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:25.520568 11266 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:25.519759 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:25.528100 11809 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:25.530423 11560 tablet_bootstrap.cc:654] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:25.531075 11561 tablet_bootstrap.cc:492] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:25.544615 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:25.554768 11484 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:25.555138 11873 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:14:25.556831 11899 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:25.558853 11899 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.555989 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 57523.1us
I20260709 13:14:25.559376 11899 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:25.583074 11333 leader_election.cc:343] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.587163 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:25.588079 11440 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:25.684677 11560 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:25.685235 11560 ts_tablet_manager.cc:1403] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.264s	user 0.031s	sys 0.031s
I20260709 13:14:25.698890 11899 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:25.699723 11560 raft_consensus.cc:359] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.700416 11560 raft_consensus.cc:385] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:25.700675 11560 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:25.701351 11560 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.705542 11561 tablet_bootstrap.cc:654] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:25.711526 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:25.711901 11825 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.726046 11560 ts_tablet_manager.cc:1434] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:14:25.731854 11560 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:25.732004 11825 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.732527 11444 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:25.732602 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:14:25.733879 11408 leader_election.cc:343] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.735291 11559 tablet_bootstrap.cc:654] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:25.735631 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:25.740170 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:25.740834 11363 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:25.760033 11264 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:25.768719 11363 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:25.771095 11903 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.771697 11903 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.776623 11875 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:25.779358 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:25.784168 11903 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.786340 11883 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:25.786849 11883 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:25.794766 11612 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:25.788561 11367 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:25.798153 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:25.804828 11483 leader_election.cc:343] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.808405 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:25.811255 11614 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:25.811656 11614 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.811944 11614 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:25.824011 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:25.787734 11883 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.834971 11745 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:25.838200 11809 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:25.842680 11559 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:25.843328 11559 ts_tablet_manager.cc:1403] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.323s	user 0.022s	sys 0.023s
W20260709 13:14:25.852247 11484 leader_election.cc:343] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.832628 11614 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.860384 11825 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.865114 11825 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:25.867177 11825 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.869242 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:25.870934 11409 leader_election.cc:343] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.879976 11614 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:25.914870 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:25.916378 11905 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:14:25.918228 11335 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:25.929919 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:25.945850 11560 tablet_bootstrap.cc:654] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:25.946394 11883 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.947001 11883 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:25.946524 11408 leader_election.cc:343] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.950325 11437 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:25.960192 11483 leader_election.cc:343] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:25.960644 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:25.951877 11408 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:26.000747 11896 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:25.983680 11883 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:25.998585 11875 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.009706 11560 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:26.010186 11560 ts_tablet_manager.cc:1403] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.279s	user 0.012s	sys 0.000s
I20260709 13:14:26.010475 11561 tablet_bootstrap.cc:492] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.010877 11561 ts_tablet_manager.cc:1403] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.480s	user 0.035s	sys 0.068s
I20260709 13:14:26.011181 11559 raft_consensus.cc:359] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.011907 11559 raft_consensus.cc:385] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.012334 11559 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.017329 11565 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:14:26.017946 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 71415.4us
I20260709 13:14:26.020165 11559 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.019222 11560 raft_consensus.cc:359] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.025048 11560 raft_consensus.cc:385] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.025349 11560 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.028107 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:26.028765 11511 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:26.029222 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:26.030030 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.023509 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:26.036276 11408 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:26.037112 11641 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:14:26.014786 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 63077.7us
I20260709 13:14:26.035931 11561 raft_consensus.cc:359] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.037496 11641 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.037807 11561 raft_consensus.cc:385] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.037882 11641 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.038136 11561 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.039047 11560 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.040568 11560 ts_tablet_manager.cc:1434] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:14:26.041558 11560 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:26.042510 11561 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.045365 11641 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.024142 11808 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.047122 11437 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
W20260709 13:14:26.047710 11409 leader_election.cc:343] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.049507 11264 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:26.072613 11483 leader_election.cc:343] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.078735 11561 ts_tablet_manager.cc:1434] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.004s
I20260709 13:14:26.084270 11825 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.084897 11825 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.086421 11641 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.086982 11825 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.089126 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:26.089715 11516 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.091215 11559 ts_tablet_manager.cc:1434] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.248s	user 0.009s	sys 0.007s
I20260709 13:14:26.092221 11559 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:26.097373 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:26.097888 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:26.099290 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:26.099888 11511 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:26.101486 11408 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:26.102262 11565 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:26.102573 11565 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.102799 11565 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:26.107784 11409 leader_election.cc:343] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:26.108327 11409 leader_election.cc:343] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.111356 11565 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.125643 11561 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.128233 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:14:26.129531 11409 leader_election.cc:343] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.129772 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:26.130326 11511 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.135699 11559 tablet_bootstrap.cc:654] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.136086 11565 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.125906 11516 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:26.142932 11408 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:26.143182 11921 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.143718 11921 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.145745 11921 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.147035 11511 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:26.147816 11565 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:26.148398 11565 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:26.149068 11565 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.151450 11408 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:26.152189 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:26.152607 11825 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:26.156482 11825 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:26.157284 11825 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.173992 11560 tablet_bootstrap.cc:654] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.178371 11561 tablet_bootstrap.cc:654] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.183538 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:26.189319 11870 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:26.189843 11870 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:26.190836 11870 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.193276 11335 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:26.194440 11903 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:26.198354 11905 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.220465 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.233803 11560 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:26.235394 11560 ts_tablet_manager.cc:1403] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.194s	user 0.010s	sys 0.009s
I20260709 13:14:26.243048 11559 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:26.243994 11559 ts_tablet_manager.cc:1403] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.152s	user 0.014s	sys 0.004s
I20260709 13:14:26.253783 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:26.255738 11921 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.272966 11560 raft_consensus.cc:359] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.273525 11560 raft_consensus.cc:385] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.273797 11560 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.274377 11560 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.276785 11560 ts_tablet_manager.cc:1434] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.001s
I20260709 13:14:26.277750 11560 tablet_bootstrap.cc:492] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:26.281725 11264 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:26.282998 11264 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:26.284253 11266 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:26.287101 11559 raft_consensus.cc:359] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.287799 11559 raft_consensus.cc:385] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.288061 11559 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.288712 11559 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.291047 11559 ts_tablet_manager.cc:1434] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.000s
I20260709 13:14:26.291742 11561 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.292192 11561 ts_tablet_manager.cc:1403] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.171s	user 0.015s	sys 0.010s
I20260709 13:14:26.293324 11560 tablet_bootstrap.cc:654] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.294363 11559 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:26.302219 11559 tablet_bootstrap.cc:654] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:26.319936 11408 consensus_peers.cc:597] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.323922 11561 raft_consensus.cc:359] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.324767 11561 raft_consensus.cc:385] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.325042 11561 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.330044 11559 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:26.330528 11559 ts_tablet_manager.cc:1403] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.013s
I20260709 13:14:26.330744 11561 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.333060 11559 raft_consensus.cc:359] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.333575 11559 raft_consensus.cc:385] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.333842 11559 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.334594 11559 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.335336 11561 ts_tablet_manager.cc:1434] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:14:26.337054 11559 ts_tablet_manager.cc:1434] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:14:26.338166 11561 tablet_bootstrap.cc:492] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.345033 11561 tablet_bootstrap.cc:654] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.345438 11559 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:26.353078 11559 tablet_bootstrap.cc:654] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.356138 11561 tablet_bootstrap.cc:492] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.356683 11561 ts_tablet_manager.cc:1403] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 13:14:26.359550 11561 raft_consensus.cc:359] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.360370 11561 raft_consensus.cc:385] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.360693 11561 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.361680 11561 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.363898 11561 ts_tablet_manager.cc:1434] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:26.364868 11561 tablet_bootstrap.cc:492] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.365221 11559 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:26.365633 11559 ts_tablet_manager.cc:1403] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.009s
I20260709 13:14:26.368134 11559 raft_consensus.cc:359] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.368777 11559 raft_consensus.cc:385] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.369037 11559 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.370707 11561 tablet_bootstrap.cc:654] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.372327 11559 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.376387 11559 ts_tablet_manager.cc:1434] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:14:26.377768 11559 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:26.382720 11560 tablet_bootstrap.cc:492] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:26.383179 11560 ts_tablet_manager.cc:1403] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.106s	user 0.012s	sys 0.020s
I20260709 13:14:26.384097 11559 tablet_bootstrap.cc:654] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.385655 11560 raft_consensus.cc:359] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.386299 11560 raft_consensus.cc:385] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.386576 11560 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.387223 11560 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.389247 11559 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:26.389714 11559 ts_tablet_manager.cc:1403] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.002s
I20260709 13:14:26.392145 11559 raft_consensus.cc:359] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.392761 11559 raft_consensus.cc:385] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.393045 11559 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.393889 11559 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.395869 11559 ts_tablet_manager.cc:1434] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:14:26.396764 11559 tablet_bootstrap.cc:492] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:26.399276 11560 ts_tablet_manager.cc:1434] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:14:26.402400 11559 tablet_bootstrap.cc:654] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.410032 11367 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.411562 11870 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.412122 11870 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.414287 11870 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:26.414528 11825 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.415073 11825 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.417121 11825 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.419266 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:26.420784 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:26.420879 11409 leader_election.cc:343] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.423830 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:26.425004 11483 leader_election.cc:343] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:26.425112 11408 leader_election.cc:343] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.425580 11408 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:26.427048 11872 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.432391 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.433049 11445 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:26.433760 11813 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.434226 11813 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.434844 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:26.436177 11926 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:26.436674 11926 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.436996 11926 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.438472 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:26.439096 11560 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:26.436209 11813 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.443270 11825 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.443626 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.443841 11825 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.444424 11445 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:26.445475 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:26.446249 11813 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.446847 11825 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.447508 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:26.448828 11408 leader_election.cc:343] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.449281 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:26.450598 11409 leader_election.cc:343] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.451040 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:26.453492 11825 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.454821 11560 tablet_bootstrap.cc:654] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.456216 11825 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:26.457242 11813 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.468974 11560 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:26.469512 11560 ts_tablet_manager.cc:1403] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 13:14:26.472282 11560 raft_consensus.cc:359] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.472972 11560 raft_consensus.cc:385] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.473278 11560 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.473973 11560 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.489002 11560 ts_tablet_manager.cc:1434] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:14:26.507539 11926 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.511685 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:14:26.513048 11483 leader_election.cc:343] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.515182 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:26.515775 11445 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.515812 11560 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:26.519376 11926 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:26.523188 11445 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:26.525203 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:26.530373 11561 tablet_bootstrap.cc:492] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.531740 11926 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:14:26.537714 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:26.532323 11561 ts_tablet_manager.cc:1403] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.168s	user 0.074s	sys 0.056s
I20260709 13:14:26.538165 11926 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:26.539039 11926 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.542163 11561 raft_consensus.cc:359] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.542887 11561 raft_consensus.cc:385] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.543196 11561 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.543977 11561 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.544996 11560 tablet_bootstrap.cc:654] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.556962 11561 ts_tablet_manager.cc:1434] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260709 13:14:26.558236 11873 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.558759 11561 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.558723 11873 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.561450 11873 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:26.563232 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.563910 11445 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:26.565270 11561 tablet_bootstrap.cc:654] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.566464 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:26.567296 11873 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:26.567778 11873 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.568099 11873 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.568795 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:26.569463 11367 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:26.567286 11559 tablet_bootstrap.cc:492] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:26.584492 11559 ts_tablet_manager.cc:1403] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.179s	user 0.019s	sys 0.035s
I20260709 13:14:26.600250 11559 raft_consensus.cc:359] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.603149 11559 raft_consensus.cc:385] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.601356 11561 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.604214 11561 ts_tablet_manager.cc:1403] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.002s
I20260709 13:14:26.605198 11559 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.606447 11559 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.607542 11265 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:26.609807 11559 ts_tablet_manager.cc:1434] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 13:14:26.610472 11561 raft_consensus.cc:359] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.611222 11561 raft_consensus.cc:385] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.611860 11561 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.612700 11561 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.615144 11561 ts_tablet_manager.cc:1434] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:14:26.616237 11561 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.616423 11559 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:26.602815 11873 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.624303 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:26.624943 11367 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.626241 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:26.626765 11445 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.627602 11560 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:26.628036 11560 ts_tablet_manager.cc:1403] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.125s	user 0.027s	sys 0.029s
I20260709 13:14:26.631361 11367 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:26.631301 11560 raft_consensus.cc:359] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.632817 11560 raft_consensus.cc:385] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.633091 11560 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.633888 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:26.633716 11560 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.634670 11870 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:26.635134 11870 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:26.635813 11560 ts_tablet_manager.cc:1434] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:14:26.636337 11559 tablet_bootstrap.cc:654] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.635941 11870 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.641763 11560 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:26.638060 11561 tablet_bootstrap.cc:654] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.656428 11445 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:26.665325 11873 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:26.666728 11560 tablet_bootstrap.cc:654] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.666409 11266 catalog_manager.cc:5697] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:26.678016 11409 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.679088 11561 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.679613 11561 ts_tablet_manager.cc:1403] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.009s
I20260709 13:14:26.680011 11808 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.680531 11808 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.682557 11808 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.682653 11561 raft_consensus.cc:359] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.683431 11561 raft_consensus.cc:385] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.683708 11561 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.685029 11561 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.685439 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.686053 11445 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:26.692996 11870 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:26.697341 11883 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:26.699250 11870 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.699759 11870 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.701627 11561 ts_tablet_manager.cc:1434] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:14:26.702006 11870 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:26.702715 11561 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.704552 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:26.705271 11808 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:26.705610 11808 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.705960 11808 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.712241 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:26.714541 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:26.715204 11515 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:26.715199 11561 tablet_bootstrap.cc:654] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
W20260709 13:14:26.729913 11409 consensus_peers.cc:597] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:26.733162 11808 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.734668 11561 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.735183 11561 ts_tablet_manager.cc:1403] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.006s
I20260709 13:14:26.739250 11561 raft_consensus.cc:359] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.739988 11561 raft_consensus.cc:385] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.740257 11561 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.741216 11565 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:26.741691 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.741313 11561 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.749161 11809 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.751724 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:26.763814 11809 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.763948 11442 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.767729 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:14:26.769222 11333 leader_election.cc:343] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.769917 11926 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:26.775193 11813 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.775771 11813 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.777473 11442 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:26.778999 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:26.778848 11813 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.774485 11809 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.779753 11511 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:26.780216 11561 ts_tablet_manager.cc:1434] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.001s
I20260709 13:14:26.783135 11561 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.786041 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:26.787065 11803 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:26.787673 11803 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:26.788264 11565 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.788735 11565 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.755309 11484 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:26.790444 11873 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.792126 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:26.792766 11363 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:26.794297 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:26.795521 11565 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.796439 11565 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:26.796824 11565 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.797225 11565 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.804054 11561 tablet_bootstrap.cc:654] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.788523 11803 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.805096 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.805711 11444 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:26.813423 11873 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.813978 11873 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.817093 11565 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.822268 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:26.822816 11363 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.827426 11265 catalog_manager.cc:5697] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:26.829411 11363 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:26.830533 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:26.831665 11883 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:26.832662 11883 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:26.833422 11883 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.736352 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:26.843652 11515 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.846449 11561 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.846903 11561 ts_tablet_manager.cc:1403] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.005s
I20260709 13:14:26.847764 11266 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:26.849864 11561 raft_consensus.cc:359] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.850482 11561 raft_consensus.cc:385] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.850759 11561 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.735252 11808 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.850541 11515 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:26.851508 11561 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.819690 11873 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:26.853538 11561 ts_tablet_manager.cc:1434] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:14:26.854063 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.854424 11561 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:26.861271 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:26.862023 11367 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:26.871739 11561 tablet_bootstrap.cc:654] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:26.873653 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:26.874311 11511 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:26.874925 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:26.888357 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:26.897921 11565 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.896745 11809 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:26.902257 11809 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.902601 11809 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.908731 11335 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:26.910445 11813 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.913072 11484 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:26.922791 11560 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:26.923442 11560 ts_tablet_manager.cc:1403] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.282s	user 0.031s	sys 0.023s
I20260709 13:14:26.928001 11561 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:26.928598 11561 ts_tablet_manager.cc:1403] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.005s
I20260709 13:14:26.930938 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:26.931586 11870 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:26.951269 11560 raft_consensus.cc:359] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.952059 11560 raft_consensus.cc:385] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.952354 11560 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.952962 11560 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.954511 11813 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.959293 11561 raft_consensus.cc:359] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.959940 11560 ts_tablet_manager.cc:1434] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.002s
I20260709 13:14:26.961273 11560 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: Bootstrap starting.
I20260709 13:14:26.956367 11809 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.960016 11561 raft_consensus.cc:385] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:26.967204 11561 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:26.968314 11561 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:26.979604 11809 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:26.980243 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:14:26.982561 11333 leader_election.cc:343] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:26.990480 11559 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:26.991051 11559 ts_tablet_manager.cc:1403] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.375s	user 0.069s	sys 0.116s
W20260709 13:14:26.993573 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 64839.4us
I20260709 13:14:26.994792 11870 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:26.997628 11519 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.001567 11517 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:27.005877 11561 ts_tablet_manager.cc:1434] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.077s	user 0.009s	sys 0.000s
I20260709 13:14:27.006345 11560 tablet_bootstrap.cc:654] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.006985 11561 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:27.008090 11825 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.008585 11825 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.010556 11825 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.012136 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:27.017956 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.010553 11559 raft_consensus.cc:359] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:27.018553 11409 leader_election.cc:343] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.018795 11559 raft_consensus.cc:385] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.019114 11559 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.019846 11559 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.021950 11559 ts_tablet_manager.cc:1434] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 13:14:27.022409 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:27.022976 11441 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:14:27.025243 11408 leader_election.cc:343] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.025765 11408 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:14:27.029507 11483 consensus_peers.cc:597] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:27.029572 11967 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.033854 11441 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:27.040589 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:27.041709 11926 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.043310 11561 tablet_bootstrap.cc:654] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.044651 11813 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:27.045842 11813 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:27.046141 11559 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:27.046870 11813 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.051756 11559 tablet_bootstrap.cc:654] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.054105 11560 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: No bootstrap required, opened a new log
I20260709 13:14:27.054641 11560 ts_tablet_manager.cc:1403] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: Time spent bootstrapping tablet: real 0.094s	user 0.009s	sys 0.002s
I20260709 13:14:27.056465 11561 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:27.057036 11561 ts_tablet_manager.cc:1403] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.001s
I20260709 13:14:27.059589 11561 raft_consensus.cc:359] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.060267 11561 raft_consensus.cc:385] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.060529 11561 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.057201 11560 raft_consensus.cc:359] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.061205 11561 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.061795 11560 raft_consensus.cc:385] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.062098 11560 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.062777 11560 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.063448 11561 ts_tablet_manager.cc:1434] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:27.064417 11561 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:27.064968 11560 ts_tablet_manager.cc:1434] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:14:27.070714 11561 tablet_bootstrap.cc:654] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.066648 11266 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:27.084086 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 43650.1us
I20260709 13:14:27.098879 11967 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.099380 11967 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.101806 11559 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:27.102341 11559 ts_tablet_manager.cc:1403] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.080s	user 0.022s	sys 0.002s
I20260709 13:14:27.103531 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:27.105197 11409 leader_election.cc:343] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.105281 11559 raft_consensus.cc:359] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.101441 11967 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.106124 11559 raft_consensus.cc:385] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.107980 11559 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.108924 11559 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.111487 11559 ts_tablet_manager.cc:1434] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:14:27.112749 11559 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:27.118499 11559 tablet_bootstrap.cc:654] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.128321 11559 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:27.128779 11559 ts_tablet_manager.cc:1403] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:14:27.131479 11559 raft_consensus.cc:359] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.132102 11559 raft_consensus.cc:385] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.132465 11559 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.133090 11559 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.136564 11559 ts_tablet_manager.cc:1434] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:14:27.137718 11559 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:27.159204 11559 tablet_bootstrap.cc:654] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.167766 11559 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:27.168215 11559 ts_tablet_manager.cc:1403] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.003s
I20260709 13:14:27.171608 11559 raft_consensus.cc:359] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.172281 11559 raft_consensus.cc:385] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.172538 11559 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.173317 11926 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.173568 11559 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.175812 11559 ts_tablet_manager.cc:1434] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:27.176828 11559 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:27.182708 11559 tablet_bootstrap.cc:654] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.195194 11559 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:27.195605 11559 ts_tablet_manager.cc:1403] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.013s
I20260709 13:14:27.197446 11559 raft_consensus.cc:359] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.197980 11559 raft_consensus.cc:385] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.198228 11559 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.198835 11559 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.201299 11559 ts_tablet_manager.cc:1434] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:14:27.202562 11559 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:27.208178 11559 tablet_bootstrap.cc:654] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.214023 11870 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.214511 11870 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.216382 11870 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:27.217908 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.218461 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:27.218497 11519 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
W20260709 13:14:27.221997 11483 leader_election.cc:343] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.224227 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.224828 11441 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:27.228477 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:27.229339 11870 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.229668 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:27.229696 11966 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:27.230309 11966 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.230614 11966 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.233211 11875 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:27.233668 11875 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.233990 11875 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.234323 11967 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.234774 11967 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.236689 11967 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.238734 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:14:27.239982 11409 leader_election.cc:343] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.242172 11966 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.242300 11875 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.244998 11813 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.245857 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.259094 11813 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.259528 11511 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 0.
I20260709 13:14:27.260334 11875 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:27.261386 11363 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.249008 11559 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:27.263471 11559 ts_tablet_manager.cc:1403] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.005s
I20260709 13:14:27.265903 11559 raft_consensus.cc:359] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.266513 11559 raft_consensus.cc:385] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.266774 11559 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.267424 11559 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.269433 11559 ts_tablet_manager.cc:1434] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:27.261041 11813 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.270366 11559 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: Bootstrap starting.
I20260709 13:14:27.271696 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.274438 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:27.274513 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:14:27.276386 11409 leader_election.cc:343] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.280607 11559 tablet_bootstrap.cc:654] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.283599 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.285168 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:27.285426 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:27.286083 11445 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:27.286294 11408 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:27.286972 11967 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:27.287308 11967 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.287631 11967 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.287626 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:27.288921 11519 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:27.292027 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:27.293059 11825 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:14:27.296401 11483 leader_election.cc:343] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.296829 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:27.297518 11926 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.302563 11559 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: No bootstrap required, opened a new log
I20260709 13:14:27.302953 11559 ts_tablet_manager.cc:1403] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.006s
I20260709 13:14:27.305505 11559 raft_consensus.cc:359] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.306173 11559 raft_consensus.cc:385] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.306432 11559 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.307070 11559 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.307864 11967 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.310024 11967 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.310662 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:27.310803 11875 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.311378 11875 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:27.312631 11409 leader_election.cc:343] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.311894 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:27.313499 11875 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:27.313530 11519 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.314589 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.315200 11441 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:27.317111 11484 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:27.318949 11970 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:27.319949 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:27.319290 11970 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.324086 11970 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.325973 11883 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.327219 11875 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.327879 11875 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:27.328707 11483 leader_election.cc:343] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.329854 11519 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:14:27.331233 11559 ts_tablet_manager.cc:1434] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:14:27.331372 11408 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:27.332612 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:27.333699 11967 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:27.334151 11967 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:27.334931 11967 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:27.336649 11483 leader_election.cc:343] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.352170 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.353018 11442 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 0.
I20260709 13:14:27.353129 11825 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.354409 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:27.355142 11926 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:27.355512 11926 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.355863 11926 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.363772 11926 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.368714 11809 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:27.369405 11926 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:27.370777 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:27.371492 11445 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.339911 11970 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.380412 11970 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:27.382387 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:27.383612 11441 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.324210 11966 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 1 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.331383 11875 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:27.406015 11445 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:27.408224 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:27.410162 11970 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:27.410816 11970 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:27.411744 11970 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.418252 11441 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:27.418555 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:27.419798 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:27.429555 11964 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.452095 11484 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:27.451853 11964 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.452720 11926 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:27.453176 11926 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:27.453998 11964 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.453938 11926 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.431838 11813 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.444209 11808 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.447394 11561 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:27.455888 11561 ts_tablet_manager.cc:1403] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.392s	user 0.122s	sys 0.172s
I20260709 13:14:27.457445 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.458062 11519 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:27.458638 11561 raft_consensus.cc:359] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.459367 11561 raft_consensus.cc:385] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.459656 11561 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.460322 11561 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.462777 11561 ts_tablet_manager.cc:1434] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:14:27.463515 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.464145 11441 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:27.465164 11333 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:27.466846 11808 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:27.467198 11808 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.467571 11808 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.469514 11561 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:27.476976 11561 tablet_bootstrap.cc:654] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.480636 11808 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.484201 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:27.484762 11441 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.485643 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:27.486219 11519 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.486552 11808 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.491362 11441 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:27.493816 11335 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:27.493173 11519 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:27.499100 11813 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:27.499614 11813 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:27.500403 11813 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.501892 11266 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:27.509792 11483 leader_election.cc:343] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:14:27.511104 11483 leader_election.cc:343] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:14:27.515245 11363 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.536573 11561 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:27.537380 11561 ts_tablet_manager.cc:1403] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.029s
I20260709 13:14:27.538154 11264 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:27.540448 11561 raft_consensus.cc:359] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.543969 11561 raft_consensus.cc:385] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.544243 11561 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.544952 11561 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.546834 11561 ts_tablet_manager.cc:1434] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:14:27.547696 11561 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:27.533468 11265 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.557042 11561 tablet_bootstrap.cc:654] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.605738 11561 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:27.606431 11561 ts_tablet_manager.cc:1403] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.059s	user 0.024s	sys 0.017s
I20260709 13:14:27.608928 11561 raft_consensus.cc:359] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.609539 11561 raft_consensus.cc:385] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.609797 11561 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.610438 11561 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.615229 11808 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.617251 11519 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.617635 11561 ts_tablet_manager.cc:1434] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:14:27.618543 11561 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:27.623183 11808 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.623728 11808 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.625356 11561 tablet_bootstrap.cc:654] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.625701 11808 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.627250 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.628851 11511 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
I20260709 13:14:27.630018 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.630620 11442 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 0.
W20260709 13:14:27.630924 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:27.631901 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:27.632973 11808 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:14:27.633335 11808 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.633646 11808 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.640986 11813 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.647704 11264 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.644296 11808 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.650656 11808 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.651355 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:27.652014 11442 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.652678 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:27.654261 11511 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.658762 11442 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:14:27.660131 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:27.661423 11813 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:14:27.662791 11813 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:27.666050 11813 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.675138 11511 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
W20260709 13:14:27.686015 11396 tablet.cc:2406] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:14:27.694396 11561 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:27.694885 11561 ts_tablet_manager.cc:1403] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.052s
I20260709 13:14:27.693713 11264 catalog_manager.cc:5697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.697395 11561 raft_consensus.cc:359] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.698040 11561 raft_consensus.cc:385] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.698310 11561 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.698922 11561 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.702587 11561 ts_tablet_manager.cc:1434] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 13:14:27.703362 11561 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: Bootstrap starting.
I20260709 13:14:27.710600 11561 tablet_bootstrap.cc:654] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: Neither blocks nor log segments found. Creating new log.
I20260709 13:14:27.712390 11808 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.712893 11808 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.714926 11808 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.716452 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.717191 11444 raft_consensus.cc:2393] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:27.719380 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.720688 11333 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:27.721699 11808 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.721941 11561 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: No bootstrap required, opened a new log
I20260709 13:14:27.722391 11561 ts_tablet_manager.cc:1403] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:14:27.724900 11561 raft_consensus.cc:359] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.725518 11561 raft_consensus.cc:385] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:14:27.725764 11561 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Initialized, Role: FOLLOWER
I20260709 13:14:27.726346 11561 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.728663 11561 ts_tablet_manager.cc:1434] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:27.730697 11808 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.754308 11363 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.761242 11967 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.771124 11967 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.808283 11808 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.808771 11808 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.810194 11808 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.810910 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.810914 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.811535 11519 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:27.811714 11442 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:14:27.812831 11335 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:27.813462 11808 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.817394 11808 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.875810 11745 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.884048 11808 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.884474 11808 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:27.886155 11808 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:27.887692 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:14:27.888495 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:27.889168 11442 raft_consensus.cc:2393] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 1: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:14:27.890316 11335 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:14:27.892299 11964 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:14:27.892376 11926 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:14:27.904047 11964 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:14:27.908139 11926 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.909893 11970 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.928054 11808 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:27.952863 11808 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.047088 11808 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:28.056427 11964 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:14:28.663568 11316 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:14:29.104048 11926 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.104522 11926 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:29.106222 11926 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:29.107023 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:14:29.107654 11366 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:29.108309 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:14:29.108916 11441 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:14:29.109045 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:29.109719 11926 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:29.110085 11926 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.110415 11926 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:29.116170 11926 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:29.117576 11926 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:29.118156 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:29.118489 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:29.118647 11366 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:29.118964 11441 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:29.124049 11366 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:14:29.124954 11441 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:14:29.126763 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:29.127764 11926 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:29.128347 11926 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:29.129082 11926 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:29.150420 11264 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 0 to 2, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.607731 11442 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:29.609097 11970 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:29.612962 11366 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:14:29.614254 11926 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:14:31.013487 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:14:31.049396 12023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:31.049463 12024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:14:31.049577 12026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:14:31.052456 10257 server_base.cc:1061] running on GCE node
I20260709 13:14:31.054090 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:14:31.054332 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:14:31.054527 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783602871054510 us; error 0 us; skew 500 ppm
I20260709 13:14:31.055085 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:14:31.058135 10257 webserver.cc:533] Webserver started at http://127.10.4.68:33913/ using document root <none> and password file <none>
I20260709 13:14:31.058578 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:14:31.058748 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:14:31.059012 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:14:31.060374 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-3-root/instance:
uuid: "b1906ab37ba541f493936685a808c8a9"
format_stamp: "Formatted at 2026-07-09 13:14:31 on dist-test-slave-gkm9"
I20260709 13:14:31.067044 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:14:31.070712 12031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:31.071493 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:14:31.071794 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-3-root
uuid: "b1906ab37ba541f493936685a808c8a9"
format_stamp: "Formatted at 2026-07-09 13:14:31 on dist-test-slave-gkm9"
I20260709 13:14:31.072062 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:14:31.094939 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:14:31.096423 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:14:31.098026 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:14:31.100929 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:14:31.101200 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:31.101492 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:14:31.101655 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:14:31.164680 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.68:37891
I20260709 13:14:31.164741 12093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.68:37891 every 8 connection(s)
I20260709 13:14:31.168910 10257 auto_leader_rebalancer-test.cc:384] add a tserver: b1906ab37ba541f493936685a808c8a9
I20260709 13:14:31.181176 12094 heartbeater.cc:344] Connected to a master server at 127.10.4.126:42409
I20260709 13:14:31.181603 12094 heartbeater.cc:461] Registering TS with master...
I20260709 13:14:31.182262 12094 heartbeater.cc:507] Master 127.10.4.126:42409 requested a full tablet report, sending...
I20260709 13:14:31.184468 11264 ts_manager.cc:194] Registered new tserver with Master: b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:14:31.186352 11264 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56588
I20260709 13:14:31.835975 11366 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:31.842929 11517 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:31.844551 11441 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:31.846009 11964 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:31.847467 11964 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:31.859386 11808 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.860734 11517 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.876020 11442 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.878571 11266 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
W20260709 13:14:31.886713 11334 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d475300e0054608a5d243d859d6609b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:31.909777 11517 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:31.922385 11444 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:31.925590 11366 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:31.926033 12022 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:31.939420 11483 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64ec4a2bcc7f4fe1851f6d77e7515fed. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:31.939921 11926 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:31.948626 11926 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.950950 11444 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.953011 11366 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.967334 11266 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.971944 11366 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:31.984083 11511 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:31.984476 11334 consensus_peers.cc:597] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e4aa2e377a0f45378e0df4a788c07ddf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:31.985221 11444 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:31.985350 11964 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:31.986619 11808 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:31.993388 11964 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.994912 11511 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:31.995496 11444 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:32.005695 11366 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.008292 11264 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:32.015682 11517 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:32.016990 11334 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b949bd9634b40af97ef4fb06ee21e9f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:32.017604 11808 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:32.025286 11444 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:32.027542 11808 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:32.050347 12104 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:32.052134 11444 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:32.053553 11511 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:32.071741 11266 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:32.189342 12094 heartbeater.cc:499] Master 127.10.4.126:42409 was elected leader, sending a full tablet report...
I20260709 13:14:32.256891 12124 ts_tablet_manager.cc:933] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:32.259696 12124 tablet_copy_client.cc:323] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:32.274648 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 3d475300e0054608a5d243d859d6609b from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:32.275228 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 3d475300e0054608a5d243d859d6609b from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-3d475300e0054608a5d243d859d6609b
I20260709 13:14:32.283550 11380 tablet_copy_source_session.cc:215] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:32.287238 12124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d475300e0054608a5d243d859d6609b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:32.303416 12124 tablet_copy_client.cc:806] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:32.304133 12124 tablet_copy_client.cc:670] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:32.311736 12124 tablet_copy_client.cc:538] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:32.321221 12124 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:32.344134 12124 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:32.345068 12124 tablet_bootstrap.cc:492] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:32.345731 12124 ts_tablet_manager.cc:1403] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.025s	user 0.025s	sys 0.000s
I20260709 13:14:32.348613 12124 raft_consensus.cc:359] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.349455 12124 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:32.350062 12124 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.353716 12124 ts_tablet_manager.cc:1434] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:14:32.355850 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-3d475300e0054608a5d243d859d6609b received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:32.356360 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-3d475300e0054608a5d243d859d6609b on tablet 3d475300e0054608a5d243d859d6609b with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:32.384547 12124 ts_tablet_manager.cc:933] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:32.398324 12124 tablet_copy_client.cc:323] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:32.400672 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet e4aa2e377a0f45378e0df4a788c07ddf from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:32.401212 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet e4aa2e377a0f45378e0df4a788c07ddf from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-e4aa2e377a0f45378e0df4a788c07ddf
I20260709 13:14:32.403959 12130 ts_tablet_manager.cc:933] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:32.407959 11380 tablet_copy_source_session.cc:215] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:32.409386 12130 tablet_copy_client.cc:323] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:32.412422 12124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4aa2e377a0f45378e0df4a788c07ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:32.426316 12124 tablet_copy_client.cc:806] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:32.426918 12124 tablet_copy_client.cc:670] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:32.427896 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:32.428503 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 64ec4a2bcc7f4fe1851f6d77e7515fed from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-64ec4a2bcc7f4fe1851f6d77e7515fed
I20260709 13:14:32.431958 12124 tablet_copy_client.cc:538] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:32.440259 12124 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:32.440761 11529 tablet_copy_source_session.cc:215] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:32.444813 12130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:32.471972 12130 tablet_copy_client.cc:806] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:32.474385 12130 tablet_copy_client.cc:670] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:32.483089 12130 tablet_copy_client.cc:538] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:32.493904 12130 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:32.497637 12134 ts_tablet_manager.cc:933] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:32.507507 12134 tablet_copy_client.cc:323] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:32.509217 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 8b949bd9634b40af97ef4fb06ee21e9f from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:32.509717 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 8b949bd9634b40af97ef4fb06ee21e9f from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-8b949bd9634b40af97ef4fb06ee21e9f
I20260709 13:14:32.516692 11380 tablet_copy_source_session.cc:215] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:32.521068 12134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b949bd9634b40af97ef4fb06ee21e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:32.522018 12124 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:32.522871 12124 tablet_bootstrap.cc:492] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:32.526850 12130 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:32.528129 12124 ts_tablet_manager.cc:1403] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.088s	user 0.030s	sys 0.008s
I20260709 13:14:32.528589 12130 tablet_bootstrap.cc:492] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:32.529357 12130 ts_tablet_manager.cc:1403] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.007s
I20260709 13:14:32.531364 12124 raft_consensus.cc:359] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.532262 12124 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:32.532934 12124 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.536413 12130 raft_consensus.cc:359] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.537421 12130 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:32.538079 12130 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.539094 12124 ts_tablet_manager.cc:1434] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:14:32.540212 12130 ts_tablet_manager.cc:1434] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 13:14:32.540930 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-e4aa2e377a0f45378e0df4a788c07ddf received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:32.541350 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-e4aa2e377a0f45378e0df4a788c07ddf on tablet e4aa2e377a0f45378e0df4a788c07ddf with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:32.544046 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-64ec4a2bcc7f4fe1851f6d77e7515fed received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:32.544454 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-64ec4a2bcc7f4fe1851f6d77e7515fed on tablet 64ec4a2bcc7f4fe1851f6d77e7515fed with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:32.546911 12134 tablet_copy_client.cc:806] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:32.547451 12134 tablet_copy_client.cc:670] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:32.551308 12134 tablet_copy_client.cc:538] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:32.559590 12134 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:32.578871 12134 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:32.579850 12134 tablet_bootstrap.cc:492] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:32.580471 12134 ts_tablet_manager.cc:1403] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.021s	user 0.023s	sys 0.001s
I20260709 13:14:32.583276 12134 raft_consensus.cc:359] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.584089 12134 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:32.584668 12134 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:32.586906 12134 ts_tablet_manager.cc:1434] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:32.588506 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-8b949bd9634b40af97ef4fb06ee21e9f received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:32.588901 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-8b949bd9634b40af97ef4fb06ee21e9f on tablet 8b949bd9634b40af97ef4fb06ee21e9f with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:32.652663 12069 raft_consensus.cc:1217] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:32.805703 12069 raft_consensus.cc:1217] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:32.915890 12069 raft_consensus.cc:1217] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:33.005798 12069 raft_consensus.cc:1217] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:33.130344 11964 raft_consensus.cc:1064] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:33.132449 11964 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.141897 12069 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.141918 11511 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.142900 11445 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.143687 11964 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.144652 12122 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.145857 12104 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.169337 12139 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.170239 11445 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.182369 12069 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.187335 11265 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.178020 11511 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.228266 11366 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.235287 11445 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.235287 12069 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.237054 12139 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.237967 11964 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.248229 11964 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.248349 12069 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.256600 11445 raft_consensus.cc:2955] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.269970 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3d475300e0054608a5d243d859d6609b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:33.276166 11267 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.313005 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d475300e0054608a5d243d859d6609b with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:33.325773 12147 tablet_replica.cc:333] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:33.326793 12147 raft_consensus.cc:2243] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:33.327746 12147 raft_consensus.cc:2272] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:33.333127 12147 ts_tablet_manager.cc:1916] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:33.347425 12147 ts_tablet_manager.cc:1929] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:33.347980 12147 log.cc:1199] T 3d475300e0054608a5d243d859d6609b P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/3d475300e0054608a5d243d859d6609b
I20260709 13:14:33.349921 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 3d475300e0054608a5d243d859d6609b (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:33.362145 12022 raft_consensus.cc:1064] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:33.364022 12022 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.369895 11445 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.370147 12069 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.371218 12107 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.372293 11366 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.372241 11926 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.373713 12022 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.389966 12107 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.397411 11444 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.411763 12069 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.415694 11266 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.431072 11366 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.446776 11511 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.453775 11442 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.455015 12153 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.455889 12069 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.459537 12022 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.468878 11442 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.473536 12122 raft_consensus.cc:1064] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:33.475953 12122 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.467590 12153 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.479522 12069 raft_consensus.cc:2955] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.490756 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:33.501824 11267 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:33.507355 12068 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.509397 12139 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.512804 11511 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.514086 12104 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.522823 12104 raft_consensus.cc:1064] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:33.523488 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 64ec4a2bcc7f4fe1851f6d77e7515fed with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:33.527982 12104 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.545599 11444 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.550465 11511 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.552803 12139 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:33.555490 12122 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:33.556726 12068 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.559689 12122 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:33.568351 11445 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:33.573926 12104 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:14:33.583227 12157 tablet_replica.cc:333] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:33.589576 12139 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.596326 11515 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.602435 11964 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.616878 12157 raft_consensus.cc:2243] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:33.621325 12068 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.618117 12157 raft_consensus.cc:2272] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:33.629865 11444 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.645015 12069 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.653581 11267 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.644416 11517 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.683247 11442 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.656888 12157 ts_tablet_manager.cc:1916] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:33.661978 11266 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.702956 12157 ts_tablet_manager.cc:1929] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:33.703444 12157 log.cc:1199] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/64ec4a2bcc7f4fe1851f6d77e7515fed
I20260709 13:14:33.705042 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 64ec4a2bcc7f4fe1851f6d77e7515fed (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:33.713366 11363 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.719198 12068 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.720767 12166 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.721823 11368 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:33.729267 11964 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.733072 12069 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.738400 11515 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.740016 12139 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.744442 12068 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.745182 11442 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:33.746778 12166 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.748332 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8b949bd9634b40af97ef4fb06ee21e9f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:33.749717 12167 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.755931 11515 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.762488 12122 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:33.762128 11267 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.770407 11442 raft_consensus.cc:2955] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.759794 12069 raft_consensus.cc:2955] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.777326 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b949bd9634b40af97ef4fb06ee21e9f with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:33.779486 12147 tablet_replica.cc:333] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:33.780491 12147 raft_consensus.cc:2243] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:33.781201 12147 raft_consensus.cc:2272] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:33.790781 12147 ts_tablet_manager.cc:1916] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:33.800125 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e4aa2e377a0f45378e0df4a788c07ddf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:33.801446 11267 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:33.820941 12147 ts_tablet_manager.cc:1929] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:33.820952 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet e4aa2e377a0f45378e0df4a788c07ddf with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:33.826478 12147 log.cc:1199] T 8b949bd9634b40af97ef4fb06ee21e9f P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/8b949bd9634b40af97ef4fb06ee21e9f
I20260709 13:14:33.827935 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 8b949bd9634b40af97ef4fb06ee21e9f (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:33.828447 12168 tablet_replica.cc:333] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:33.829360 12168 raft_consensus.cc:2243] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:33.829947 12168 raft_consensus.cc:2272] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:33.833760 12168 ts_tablet_manager.cc:1916] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:33.847467 12168 ts_tablet_manager.cc:1929] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:33.847879 12168 log.cc:1199] T e4aa2e377a0f45378e0df4a788c07ddf P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/e4aa2e377a0f45378e0df4a788c07ddf
I20260709 13:14:33.849570 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet e4aa2e377a0f45378e0df4a788c07ddf (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:34.884164 11515 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:34.890558 11368 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:34.890545 11483 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47922bb832834e72bc79e1e8159e6c1f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:34.892446 11444 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:34.892310 12107 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:34.893579 11926 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:34.902546 12022 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:34.905046 11363 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:34.912004 11442 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:34.920228 11266 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:34.933049 11515 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:14:34.956693 11483 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae095c448d6e4a3ab9eaecd9695887a7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:34.958038 11445 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:34.958475 11363 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:34.959594 12175 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:34.960598 12107 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:34.970602 12022 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:34.974251 11368 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:34.975379 11445 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:34.990350 11368 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:34.989989 11267 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:34.997995 11442 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:34.998127 11334 consensus_peers.cc:597] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 435f2e223e784d188897ac4adbc34211. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:34.997747 11515 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:34.999488 12167 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:35.000548 12122 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:35.013165 12181 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.015077 11517 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.017307 11442 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.026814 11515 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.033411 11442 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:35.033999 11483 consensus_peers.cc:597] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 741facf7b8264a10bc5c3431415fc34a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:35.034904 11926 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:35.035578 11368 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:35.037104 11926 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:35.038801 11266 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.045717 11926 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.048257 11442 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.085407 11265 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.097757 11368 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:35.284024 12187 ts_tablet_manager.cc:933] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:35.285655 12187 tablet_copy_client.cc:323] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:35.287010 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 47922bb832834e72bc79e1e8159e6c1f from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:35.287560 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 47922bb832834e72bc79e1e8159e6c1f from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-47922bb832834e72bc79e1e8159e6c1f
I20260709 13:14:35.294446 11529 tablet_copy_source_session.cc:215] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:35.298089 12187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47922bb832834e72bc79e1e8159e6c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:35.311255 12187 tablet_copy_client.cc:806] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:35.311815 12187 tablet_copy_client.cc:670] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:35.315367 12187 tablet_copy_client.cc:538] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:35.322117 12187 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:35.344170 12187 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.345003 12187 tablet_bootstrap.cc:492] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:35.345620 12187 ts_tablet_manager.cc:1403] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.009s
I20260709 13:14:35.348534 12187 raft_consensus.cc:359] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.349340 12187 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:35.349963 12187 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.357992 12187 ts_tablet_manager.cc:1434] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.010s
I20260709 13:14:35.360242 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-47922bb832834e72bc79e1e8159e6c1f received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:35.360677 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-47922bb832834e72bc79e1e8159e6c1f on tablet 47922bb832834e72bc79e1e8159e6c1f with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:35.475752 12187 ts_tablet_manager.cc:933] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:35.482650 12187 tablet_copy_client.cc:323] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:35.490260 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 741facf7b8264a10bc5c3431415fc34a from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:35.493794 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 741facf7b8264a10bc5c3431415fc34a from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-741facf7b8264a10bc5c3431415fc34a
I20260709 13:14:35.556273 11529 tablet_copy_source_session.cc:215] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:35.558929 12198 ts_tablet_manager.cc:933] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:35.559764 12199 ts_tablet_manager.cc:933] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:35.596836 12198 tablet_copy_client.cc:323] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:35.599134 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 435f2e223e784d188897ac4adbc34211 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:35.599726 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 435f2e223e784d188897ac4adbc34211 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-435f2e223e784d188897ac4adbc34211
I20260709 13:14:35.606442 12199 tablet_copy_client.cc:323] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:35.608253 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet ae095c448d6e4a3ab9eaecd9695887a7 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:35.608834 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet ae095c448d6e4a3ab9eaecd9695887a7 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-ae095c448d6e4a3ab9eaecd9695887a7
I20260709 13:14:35.615607 11380 tablet_copy_source_session.cc:215] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:35.617920 12187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 741facf7b8264a10bc5c3431415fc34a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:35.619825 12198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435f2e223e784d188897ac4adbc34211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:35.634579 11528 tablet_copy_source_session.cc:215] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:35.646940 12199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae095c448d6e4a3ab9eaecd9695887a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:35.654392 12187 tablet_copy_client.cc:806] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:35.655148 12187 tablet_copy_client.cc:670] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:35.665184 12187 tablet_copy_client.cc:538] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:35.678459 12068 raft_consensus.cc:1217] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:35.681414 12187 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:35.690313 12199 tablet_copy_client.cc:806] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:35.690977 12199 tablet_copy_client.cc:670] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:35.700130 12199 tablet_copy_client.cc:538] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:35.716156 12198 tablet_copy_client.cc:806] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:35.726652 12198 tablet_copy_client.cc:670] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:35.726032 12199 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:35.731269 12198 tablet_copy_client.cc:538] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:35.756460 12198 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:35.769743 12187 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.770807 12187 tablet_bootstrap.cc:492] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:35.771581 12187 ts_tablet_manager.cc:1403] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.091s	user 0.025s	sys 0.018s
I20260709 13:14:35.775450 12187 raft_consensus.cc:359] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.776728 12187 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:35.777539 12187 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.783650 12187 ts_tablet_manager.cc:1434] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 13:14:35.785550 11528 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-741facf7b8264a10bc5c3431415fc34a received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:35.786039 11528 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-741facf7b8264a10bc5c3431415fc34a on tablet 741facf7b8264a10bc5c3431415fc34a with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:35.794505 12199 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.795698 12199 tablet_bootstrap.cc:492] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:35.796614 12199 ts_tablet_manager.cc:1403] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.071s	user 0.021s	sys 0.009s
I20260709 13:14:35.800458 12199 raft_consensus.cc:359] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.801544 12198 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:35.801641 12199 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:35.802925 12198 tablet_bootstrap.cc:492] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:35.802466 12199 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.803894 12198 ts_tablet_manager.cc:1403] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.048s	user 0.024s	sys 0.007s
I20260709 13:14:35.805716 12199 ts_tablet_manager.cc:1434] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:14:35.806890 12198 raft_consensus.cc:359] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.807744 12198 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:35.808362 12198 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:35.810846 12198 ts_tablet_manager.cc:1434] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:14:35.811041 11528 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-ae095c448d6e4a3ab9eaecd9695887a7 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:35.811487 11528 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-ae095c448d6e4a3ab9eaecd9695887a7 on tablet ae095c448d6e4a3ab9eaecd9695887a7 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:35.812546 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-435f2e223e784d188897ac4adbc34211 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:35.813026 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-435f2e223e784d188897ac4adbc34211 on tablet 435f2e223e784d188897ac4adbc34211 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:35.914968 12068 raft_consensus.cc:1217] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:36.044757 12068 raft_consensus.cc:1217] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:36.078403 12068 raft_consensus.cc:1217] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:36.226898 12193 raft_consensus.cc:1064] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:36.229624 12193 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.237119 12068 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.240335 11444 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.242223 12107 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:36.248819 12022 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:14:36.255949 11363 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.266584 12107 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:36.278457 12022 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.282371 11442 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.303577 12068 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.305944 11265 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.311451 11363 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.343729 11517 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.348210 12068 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.348619 11442 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.349542 12107 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.351550 12193 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.359342 12207 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.362596 12068 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.369379 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 47922bb832834e72bc79e1e8159e6c1f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:36.370764 11444 raft_consensus.cc:2955] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.386060 11264 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.396229 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 47922bb832834e72bc79e1e8159e6c1f with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:36.397961 12216 tablet_replica.cc:333] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:36.398955 12216 raft_consensus.cc:2243] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:36.399664 12216 raft_consensus.cc:2272] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:36.403923 12216 ts_tablet_manager.cc:1916] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:36.410523 12107 raft_consensus.cc:1064] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:36.412657 12107 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.427162 12216 ts_tablet_manager.cc:1929] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:36.427802 12216 log.cc:1199] T 47922bb832834e72bc79e1e8159e6c1f P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/47922bb832834e72bc79e1e8159e6c1f
I20260709 13:14:36.429075 12068 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.429478 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 47922bb832834e72bc79e1e8159e6c1f (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:36.431169 11363 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.431631 11444 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.431949 12207 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:36.432878 12022 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:36.434331 12022 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:36.449004 12022 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.450215 11366 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.472675 12068 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.478313 11444 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.484464 11266 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.503896 11519 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.507380 12193 raft_consensus.cc:1064] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:36.509987 12193 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.516343 12068 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.516777 11366 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.518378 12207 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.521890 11442 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.522270 12207 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.523640 12208 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:36.526484 12208 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.524194 12068 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.533880 11363 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.537524 11368 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.549039 12107 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:36.554141 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 741facf7b8264a10bc5c3431415fc34a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:36.545697 12068 raft_consensus.cc:2955] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.559525 11266 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.570451 12022 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:14:36.584642 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 741facf7b8264a10bc5c3431415fc34a with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:36.616981 12167 raft_consensus.cc:1064] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:36.643942 12167 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.716274 12223 tablet_replica.cc:333] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:36.725802 12223 raft_consensus.cc:2243] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:36.724483 12068 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:14:36.717671 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 115892us
I20260709 13:14:36.729668 11441 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.735646 11517 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:36.740886 12068 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.747426 12192 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:36.741521 12223 raft_consensus.cc:2272] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:36.756376 12223 ts_tablet_manager.cc:1916] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:36.756855 11363 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.759943 11267 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.739492 12207 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.782922 12167 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:36.785632 12227 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:36.798564 11441 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.811964 11517 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.812724 12223 ts_tablet_manager.cc:1929] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:36.824093 12223 log.cc:1199] T 741facf7b8264a10bc5c3431415fc34a P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/741facf7b8264a10bc5c3431415fc34a
I20260709 13:14:36.827139 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 741facf7b8264a10bc5c3431415fc34a (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:36.814921 12192 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.822296 11519 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.849349 12068 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.851255 12069 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.852934 11366 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.854046 12022 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.854797 12229 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.865777 11266 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.871557 11441 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.874099 12022 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.883031 12069 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.886031 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ae095c448d6e4a3ab9eaecd9695887a7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:36.888283 11363 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:36.898326 11267 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.899959 11363 raft_consensus.cc:2955] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.915354 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet ae095c448d6e4a3ab9eaecd9695887a7 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:36.916862 12223 tablet_replica.cc:333] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:36.917843 12223 raft_consensus.cc:2243] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:36.918646 12223 raft_consensus.cc:2272] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:36.919713 12067 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.922096 12167 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.923197 12223 ts_tablet_manager.cc:1916] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:36.927390 11444 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:36.938169 12233 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.950799 12223 ts_tablet_manager.cc:1929] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:36.953203 12223 log.cc:1199] T ae095c448d6e4a3ab9eaecd9695887a7 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/ae095c448d6e4a3ab9eaecd9695887a7
I20260709 13:14:36.955140 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet ae095c448d6e4a3ab9eaecd9695887a7 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:36.955933 12227 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.957863 11441 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.971376 12069 raft_consensus.cc:2955] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.983386 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 435f2e223e784d188897ac4adbc34211 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:36.984078 11265 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:36.993860 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 435f2e223e784d188897ac4adbc34211 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:37.009167 12234 tablet_replica.cc:333] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:37.009954 12234 raft_consensus.cc:2243] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:37.010473 12234 raft_consensus.cc:2272] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:37.014446 12234 ts_tablet_manager.cc:1916] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:37.026865 12234 ts_tablet_manager.cc:1929] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:37.027158 12234 log.cc:1199] T 435f2e223e784d188897ac4adbc34211 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/435f2e223e784d188897ac4adbc34211
I20260709 13:14:37.028419 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 435f2e223e784d188897ac4adbc34211 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:38.207160 11515 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.212929 11363 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:38.212988 11440 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:38.213258 11483 consensus_peers.cc:597] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6e7edf5f82e49158a4023ea61d78284. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:38.214241 12235 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.215217 12246 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.221828 12235 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.223551 11363 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.226212 11440 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.241500 11515 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.241722 11266 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.248863 11443 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:38.250239 12236 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:38.251969 11483 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c753736d0d7f4578b8935b855a6b36e3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:38.253407 11363 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:38.259745 12235 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:38.263312 12235 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.264916 11440 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.277801 11363 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.279129 11366 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:14:38.285339 11334 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ad5a541019548a3b543619a92b420b1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:38.286756 11445 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:38.287474 11515 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:38.288511 12238 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.289732 12227 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.295213 12227 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.296886 11445 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.313580 11511 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.320580 11265 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:38.339668 11517 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.342341 11440 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:38.343919 12245 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:38.344864 11483 consensus_peers.cc:597] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78b9049e00f94cc1912970768037cf24. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:38.345454 11366 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:38.346937 12245 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:38.348562 11266 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:38.366601 12235 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.370983 11440 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.370644 11366 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:38.384722 11265 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:38.672437 12264 ts_tablet_manager.cc:933] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:38.674552 12264 tablet_copy_client.cc:323] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:38.676168 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet e6e7edf5f82e49158a4023ea61d78284 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:38.676690 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet e6e7edf5f82e49158a4023ea61d78284 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-e6e7edf5f82e49158a4023ea61d78284
I20260709 13:14:38.685252 11528 tablet_copy_source_session.cc:215] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:38.688195 12264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e7edf5f82e49158a4023ea61d78284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:38.702631 12264 tablet_copy_client.cc:806] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:38.703200 12264 tablet_copy_client.cc:670] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:38.718797 12264 tablet_copy_client.cc:538] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:38.727782 12264 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:38.747222 12264 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:38.748142 12264 tablet_bootstrap.cc:492] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:38.748814 12264 ts_tablet_manager.cc:1403] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:14:38.751574 12264 raft_consensus.cc:359] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.752370 12264 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:38.752985 12264 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.755663 12264 ts_tablet_manager.cc:1434] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:14:38.757607 11528 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-e6e7edf5f82e49158a4023ea61d78284 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:38.758008 11528 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-e6e7edf5f82e49158a4023ea61d78284 on tablet e6e7edf5f82e49158a4023ea61d78284 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:38.762527 12264 ts_tablet_manager.cc:933] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:38.770146 12264 tablet_copy_client.cc:323] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:38.771795 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 4ad5a541019548a3b543619a92b420b1 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:38.772346 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 4ad5a541019548a3b543619a92b420b1 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-4ad5a541019548a3b543619a92b420b1
I20260709 13:14:38.779660 11380 tablet_copy_source_session.cc:215] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:38.782956 12264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad5a541019548a3b543619a92b420b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:38.783464 12268 ts_tablet_manager.cc:933] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:38.785626 12268 tablet_copy_client.cc:323] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:38.787197 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet c753736d0d7f4578b8935b855a6b36e3 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:38.787735 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet c753736d0d7f4578b8935b855a6b36e3 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-c753736d0d7f4578b8935b855a6b36e3
I20260709 13:14:38.795804 11528 tablet_copy_source_session.cc:215] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:38.797436 12264 tablet_copy_client.cc:806] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:38.798062 12264 tablet_copy_client.cc:670] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:38.798859 12268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c753736d0d7f4578b8935b855a6b36e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:38.804395 12264 tablet_copy_client.cc:538] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:38.812873 12264 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:38.814656 12268 tablet_copy_client.cc:806] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:38.815270 12268 tablet_copy_client.cc:670] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:38.818938 12268 tablet_copy_client.cc:538] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:38.827937 12268 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:38.853283 12264 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:38.854362 12264 tablet_bootstrap.cc:492] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:38.855132 12264 ts_tablet_manager.cc:1403] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.012s
I20260709 13:14:38.867417 12264 raft_consensus.cc:359] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.868492 12264 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:38.870676 12264 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.874706 12264 ts_tablet_manager.cc:1434] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:14:38.876385 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-4ad5a541019548a3b543619a92b420b1 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:38.876878 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-4ad5a541019548a3b543619a92b420b1 on tablet 4ad5a541019548a3b543619a92b420b1 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:38.880017 12264 ts_tablet_manager.cc:933] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:38.885390 12268 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:38.886324 12268 tablet_bootstrap.cc:492] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:38.887017 12268 ts_tablet_manager.cc:1403] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.029s
I20260709 13:14:38.888025 12264 tablet_copy_client.cc:323] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:38.890048 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 78b9049e00f94cc1912970768037cf24 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:38.890303 12268 raft_consensus.cc:359] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.891240 12268 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:38.890537 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 78b9049e00f94cc1912970768037cf24 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-78b9049e00f94cc1912970768037cf24
I20260709 13:14:38.891961 12268 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.894564 12268 ts_tablet_manager.cc:1434] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:14:38.896138 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-c753736d0d7f4578b8935b855a6b36e3 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:38.896529 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-c753736d0d7f4578b8935b855a6b36e3 on tablet c753736d0d7f4578b8935b855a6b36e3 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:38.898486 11528 tablet_copy_source_session.cc:215] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:38.903709 12264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b9049e00f94cc1912970768037cf24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:38.916177 12264 tablet_copy_client.cc:806] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:38.916658 12264 tablet_copy_client.cc:670] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:38.919812 12264 tablet_copy_client.cc:538] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:38.926131 12264 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:38.941735 12264 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:38.942617 12264 tablet_bootstrap.cc:492] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:38.943252 12264 ts_tablet_manager.cc:1403] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.017s	user 0.018s	sys 0.000s
I20260709 13:14:38.945525 12264 raft_consensus.cc:359] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.946089 12264 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:38.946599 12264 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:38.948616 12264 ts_tablet_manager.cc:1434] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:14:38.950091 11528 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-78b9049e00f94cc1912970768037cf24 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:38.950477 11528 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-78b9049e00f94cc1912970768037cf24 on tablet 78b9049e00f94cc1912970768037cf24 with peer b1906ab37ba541f493936685a808c8a9
W20260709 13:14:39.022951 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08001dbd00 after lost signal to thread 10260
W20260709 13:14:39.023885 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08001fd1e0 after lost signal to thread 11302
W20260709 13:14:39.024696 11466 debug-util.cc:398] Leaking SignalData structure 0x7b0800157660 after lost signal to thread 11394
W20260709 13:14:39.025475 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08002eca60 after lost signal to thread 11469
W20260709 13:14:39.026191 11466 debug-util.cc:398] Leaking SignalData structure 0x7b0800154a40 after lost signal to thread 11543
W20260709 13:14:39.026939 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08002e35a0 after lost signal to thread 12093
I20260709 13:14:39.088385 12067 raft_consensus.cc:1217] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:39.160321 12067 raft_consensus.cc:1217] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:39.286110 12067 raft_consensus.cc:1217] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:39.402647 12067 raft_consensus.cc:1217] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:39.476522 12245 raft_consensus.cc:1064] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:39.478941 12245 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:39.493757 12069 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.495832 11442 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.496630 11364 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.495831 12229 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:14:39.498478 12245 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.499517 12246 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.550503 12229 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.564656 11442 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.585047 11266 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.595930 12069 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.624095 11364 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:14:39.690994 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52306.5us
I20260709 13:14:39.710469 11511 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:39.727185 12069 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:39.727919 11442 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:39.732322 12283 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:39.734669 12283 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:39.739650 12246 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.749158 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e6e7edf5f82e49158a4023ea61d78284 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:39.748358 11442 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.759572 11264 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:39.764065 12227 raft_consensus.cc:1064] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:39.766553 12227 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:39.776963 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet e6e7edf5f82e49158a4023ea61d78284 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:39.775694 12069 raft_consensus.cc:2955] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.779556 12067 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.781088 12238 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.797813 12287 tablet_replica.cc:333] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:39.798720 12287 raft_consensus.cc:2243] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:39.799289 12287 raft_consensus.cc:2272] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:39.802983 12287 ts_tablet_manager.cc:1916] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:39.799300 11511 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.807606 12238 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.810019 11442 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.819849 12287 ts_tablet_manager.cc:1929] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:39.820372 12287 log.cc:1199] T e6e7edf5f82e49158a4023ea61d78284 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/e6e7edf5f82e49158a4023ea61d78284
I20260709 13:14:39.822014 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet e6e7edf5f82e49158a4023ea61d78284 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:39.821105 12227 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:14:39.869292 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 44843.7us
I20260709 13:14:39.870814 12227 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.874377 11445 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.881356 11511 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.893554 12067 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.904597 11264 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:39.908756 12283 raft_consensus.cc:1064] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:39.911273 12283 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:39.917989 11364 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:39.937268 12069 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:39.940155 12291 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:14:39.948750 12066 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.950687 12283 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.952736 11445 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.955390 12283 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.962575 12069 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.965427 11364 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:39.969044 12280 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:39.969952 12227 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:39.971103 12280 raft_consensus.cc:1064] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:39.983465 12280 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:39.973713 11442 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:39.990417 12167 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:40.000095 11265 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.001857 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ad5a541019548a3b543619a92b420b1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:40.004562 12245 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.010638 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ad5a541019548a3b543619a92b420b1 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:40.022490 11366 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.012668 11442 raft_consensus.cc:2955] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.033612 12298 tablet_replica.cc:333] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:40.042017 12298 raft_consensus.cc:2243] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.043897 12298 raft_consensus.cc:2272] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.043730 11264 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.046909 11437 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.048995 12298 ts_tablet_manager.cc:1916] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:40.100687 12298 ts_tablet_manager.cc:1929] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:40.101166 12298 log.cc:1199] T 4ad5a541019548a3b543619a92b420b1 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/4ad5a541019548a3b543619a92b420b1
I20260709 13:14:40.106421 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 4ad5a541019548a3b543619a92b420b1 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
W20260709 13:14:40.128638 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 61394.2us
I20260709 13:14:40.136945 12067 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:40.136849 11364 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:40.138329 12069 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.139598 12283 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:40.140453 11437 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:40.149003 12283 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:40.154134 12283 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:40.164247 11511 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:40.184640 12067 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:40.187049 11366 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:40.193931 12246 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:40.201243 12235 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:40.208964 12283 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.221865 11364 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.228389 11442 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.233007 12245 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.234949 11264 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:40.250578 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 78b9049e00f94cc1912970768037cf24 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:40.252882 12069 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.253119 12067 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.266975 11265 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.274343 11366 raft_consensus.cc:2955] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.278019 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 78b9049e00f94cc1912970768037cf24 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:40.323468 12302 tablet_replica.cc:333] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:40.324582 12302 raft_consensus.cc:2243] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.325280 12302 raft_consensus.cc:2272] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.326052 11511 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:40.329936 12302 ts_tablet_manager.cc:1916] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:40.334194 12069 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:40.334937 11442 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:40.336500 12245 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:40.337311 12235 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:40.344628 12246 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.346915 12069 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.363840 11264 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.365645 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c753736d0d7f4578b8935b855a6b36e3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:40.368556 11442 raft_consensus.cc:2955] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:40.398955 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet c753736d0d7f4578b8935b855a6b36e3 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:40.414747 12303 tablet_replica.cc:333] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:40.416288 12303 raft_consensus.cc:2243] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.416973 12303 raft_consensus.cc:2272] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.420131 12303 ts_tablet_manager.cc:1916] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:40.421868 12302 ts_tablet_manager.cc:1929] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:40.422278 12302 log.cc:1199] T 78b9049e00f94cc1912970768037cf24 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/78b9049e00f94cc1912970768037cf24
I20260709 13:14:40.431346 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 78b9049e00f94cc1912970768037cf24 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:40.435676 12303 ts_tablet_manager.cc:1929] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:40.436121 12303 log.cc:1199] T c753736d0d7f4578b8935b855a6b36e3 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/c753736d0d7f4578b8935b855a6b36e3
I20260709 13:14:40.437702 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet c753736d0d7f4578b8935b855a6b36e3 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:41.523598 11440 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:41.532711 11511 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:41.535141 12304 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:41.538018 11363 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:41.543748 12304 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:41.547648 11409 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11c8fff0a9ad4f4c9b41c65f74776fd6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.552618 12247 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.554792 11515 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.559340 11363 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.578667 11267 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.579332 11515 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:41.593686 11366 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:41.593856 11483 consensus_peers.cc:597] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 116f1226682347599c239ad3bee95b41. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.594732 11440 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:41.596184 12305 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:41.602432 12245 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:41.605693 12245 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.608944 11440 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.644649 11363 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.656798 11264 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.661267 11511 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:14:41.680728 11483 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 887116b7f3e447028e317f03386bf8d0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.683040 11440 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:41.684444 12245 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:41.684888 11363 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:41.686295 12322 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:41.700840 12322 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.705317 11440 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.715471 11366 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.723976 11265 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.737757 11511 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:14:41.764001 11483 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6afc1298d03c4fa6a7ce8e41231e8cf4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.764009 11363 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:41.765136 11440 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:41.766623 12322 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:41.773543 12322 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:41.798712 12321 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.799959 11363 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.800145 11442 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.828042 11264 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:41.989522 12329 ts_tablet_manager.cc:933] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:41.999472 12329 tablet_copy_client.cc:323] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:14:42.028574 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:14:42.029178 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-11c8fff0a9ad4f4c9b41c65f74776fd6
I20260709 13:14:42.057847 11455 tablet_copy_source_session.cc:215] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:42.061556 12329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.084666 12329 tablet_copy_client.cc:806] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:42.085603 12329 tablet_copy_client.cc:670] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:42.089619 12329 tablet_copy_client.cc:538] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:42.102072 12329 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:42.109105 12332 ts_tablet_manager.cc:933] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:42.122135 12332 tablet_copy_client.cc:323] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:42.124080 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 116f1226682347599c239ad3bee95b41 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:42.124640 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 116f1226682347599c239ad3bee95b41 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-116f1226682347599c239ad3bee95b41
I20260709 13:14:42.132977 11528 tablet_copy_source_session.cc:215] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:42.135521 12329 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:42.136703 12329 tablet_bootstrap.cc:492] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:42.141126 12329 ts_tablet_manager.cc:1403] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.007s
I20260709 13:14:42.144121 12329 raft_consensus.cc:359] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.144989 12329 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:42.145658 12329 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.157037 12329 ts_tablet_manager.cc:1434] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.005s
I20260709 13:14:42.159919 12332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116f1226682347599c239ad3bee95b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.159842 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-11c8fff0a9ad4f4c9b41c65f74776fd6 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:14:42.163733 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-11c8fff0a9ad4f4c9b41c65f74776fd6 on tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:42.185811 12332 tablet_copy_client.cc:806] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:42.186465 12332 tablet_copy_client.cc:670] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:42.190281 12329 ts_tablet_manager.cc:933] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:42.210988 12329 tablet_copy_client.cc:323] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:42.212795 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 887116b7f3e447028e317f03386bf8d0 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:42.213543 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 887116b7f3e447028e317f03386bf8d0 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-887116b7f3e447028e317f03386bf8d0
I20260709 13:14:42.226477 11528 tablet_copy_source_session.cc:215] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:42.230793 12329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887116b7f3e447028e317f03386bf8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.238803 12332 tablet_copy_client.cc:538] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:42.257906 12332 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:42.263805 12329 tablet_copy_client.cc:806] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:42.264532 12329 tablet_copy_client.cc:670] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:42.268493 12329 tablet_copy_client.cc:538] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:42.294528 12332 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:42.295591 12332 tablet_bootstrap.cc:492] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:42.296253 12329 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:42.296324 12332 ts_tablet_manager.cc:1403] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.011s
I20260709 13:14:42.299490 12332 raft_consensus.cc:359] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.300362 12332 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:42.301059 12332 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.303785 12332 ts_tablet_manager.cc:1434] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:42.305624 11528 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-116f1226682347599c239ad3bee95b41 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:42.306002 11528 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-116f1226682347599c239ad3bee95b41 on tablet 116f1226682347599c239ad3bee95b41 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:42.319118 12332 ts_tablet_manager.cc:933] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:42.344360 12332 tablet_copy_client.cc:323] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:42.346117 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:42.346650 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-6afc1298d03c4fa6a7ce8e41231e8cf4
I20260709 13:14:42.402536 11528 tablet_copy_source_session.cc:215] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:42.408968 12332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:42.444856 12329 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:42.449666 12329 tablet_bootstrap.cc:492] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:42.450536 12329 ts_tablet_manager.cc:1403] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.155s	user 0.020s	sys 0.030s
I20260709 13:14:42.453856 12329 raft_consensus.cc:359] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.454785 12329 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:42.455482 12329 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.470858 12332 tablet_copy_client.cc:806] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:42.473765 12332 tablet_copy_client.cc:670] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:42.479136 12332 tablet_copy_client.cc:538] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:42.479745 12329 ts_tablet_manager.cc:1434] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.004s
I20260709 13:14:42.489473 11528 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-887116b7f3e447028e317f03386bf8d0 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:42.489917 11528 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-887116b7f3e447028e317f03386bf8d0 on tablet 887116b7f3e447028e317f03386bf8d0 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:42.491008 12332 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:42.532047 12332 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:42.533129 12332 tablet_bootstrap.cc:492] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:42.533879 12332 ts_tablet_manager.cc:1403] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.012s
I20260709 13:14:42.538045 12332 raft_consensus.cc:359] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.538997 12332 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:42.539898 12332 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:42.542593 12332 ts_tablet_manager.cc:1434] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:14:42.549170 11528 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-6afc1298d03c4fa6a7ce8e41231e8cf4 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:42.549623 11528 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-6afc1298d03c4fa6a7ce8e41231e8cf4 on tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:42.613198 12069 raft_consensus.cc:1217] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:42.625613 12069 raft_consensus.cc:1217] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:42.649935 12069 raft_consensus.cc:1217] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:42.755096 12069 raft_consensus.cc:1217] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:43.141285 12322 raft_consensus.cc:1064] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:43.144991 12322 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.153085 12304 raft_consensus.cc:1064] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:43.157828 12304 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.171070 12069 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.175429 12245 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.176705 12246 raft_consensus.cc:1064] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:43.178757 12246 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.210919 11363 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.217132 11363 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.219416 12304 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.219776 12069 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.221334 12349 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.236413 11442 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.258855 11364 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.259063 12069 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.265897 11442 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.267230 12342 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:43.270167 11517 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.273463 12342 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:43.275282 12246 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:43.278175 12346 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.285378 12346 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.300499 12246 raft_consensus.cc:1064] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:43.303264 12246 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.315384 12348 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:14:43.320240 12304 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.328588 12067 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.384971 11364 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.395457 12069 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.402020 11366 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.402854 11264 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.417995 12346 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.422713 12246 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.425009 11445 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:43.428768 12322 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.421299 11515 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.441960 12067 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.443320 11364 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.443915 11366 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.450883 12322 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.452677 12245 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:43.457846 11266 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:43.465651 11442 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.473016 11440 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.481743 12069 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.508641 11445 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.507264 12245 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.526244 12067 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.531982 11368 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.540612 11515 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.553547 11437 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.561277 12069 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.562971 12346 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:43.564209 11368 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.564553 11364 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.565982 12340 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:43.566578 12069 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.569305 12349 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:43.573150 12346 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:43.580528 12348 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.585496 12246 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.579815 11364 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.583068 11268 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.595386 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 116f1226682347599c239ad3bee95b41 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:43.599180 12069 raft_consensus.cc:2955] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.603724 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:43.611142 12067 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.627722 11364 raft_consensus.cc:2955] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.637224 11262 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:43.660250 11267 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:14:43.665052 11254 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:14:43.670289 11515 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.676926 12067 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.677917 11517 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:43.678416 12245 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:43.678341 11437 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.681767 12246 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:43.683548 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:43.685693 11262 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.692909 12069 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.694382 12245 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:43.695227 11440 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:43.696687 12322 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:43.716207 12321 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.718247 11440 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.719530 12346 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.722334 12067 raft_consensus.cc:2955] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.728034 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 116f1226682347599c239ad3bee95b41 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:43.730615 12069 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.733383 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 887116b7f3e447028e317f03386bf8d0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:43.737694 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:43.740000 11437 raft_consensus.cc:2955] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.744601 12372 tablet_replica.cc:333] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:43.749446 12372 raft_consensus.cc:2243] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.752123 12372 raft_consensus.cc:2272] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.767925 11262 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:43.781826 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 887116b7f3e447028e317f03386bf8d0 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:43.798599 12372 ts_tablet_manager.cc:1916] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:43.801654 11267 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:43.807664 12371 tablet_replica.cc:333] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:43.808645 12371 raft_consensus.cc:2243] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.809279 12371 raft_consensus.cc:2272] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.811087 12373 tablet_replica.cc:333] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:43.812124 12373 raft_consensus.cc:2243] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.812831 12373 raft_consensus.cc:2272] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.815747 12373 ts_tablet_manager.cc:1916] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:43.822425 12372 ts_tablet_manager.cc:1929] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:43.823011 12372 log.cc:1199] T 116f1226682347599c239ad3bee95b41 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/116f1226682347599c239ad3bee95b41
I20260709 13:14:43.824880 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:43.826052 12371 ts_tablet_manager.cc:1916] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:43.825554 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 116f1226682347599c239ad3bee95b41 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:43.846900 12373 ts_tablet_manager.cc:1929] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:43.846900 12371 ts_tablet_manager.cc:1929] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:43.847488 12373 log.cc:1199] T 887116b7f3e447028e317f03386bf8d0 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/887116b7f3e447028e317f03386bf8d0
I20260709 13:14:43.847621 12371 log.cc:1199] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/11c8fff0a9ad4f4c9b41c65f74776fd6
I20260709 13:14:43.849042 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 11c8fff0a9ad4f4c9b41c65f74776fd6 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:43.849243 12373 tablet_replica.cc:333] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:43.849154 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 887116b7f3e447028e317f03386bf8d0 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:43.850067 12373 raft_consensus.cc:2243] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.850538 12373 raft_consensus.cc:2272] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.852988 12373 ts_tablet_manager.cc:1916] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:43.864128 12373 ts_tablet_manager.cc:1929] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:43.864410 12373 log.cc:1199] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/6afc1298d03c4fa6a7ce8e41231e8cf4
I20260709 13:14:43.865478 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 6afc1298d03c4fa6a7ce8e41231e8cf4 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:44.892424 11364 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:14:44.897972 11334 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7074ddeb0b7b4c1395b798352ff7db0a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:44.899677 11440 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:44.899788 11515 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:44.901028 12238 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:44.901624 12370 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:44.912036 12370 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:44.919749 11515 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:44.921320 11445 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:44.934304 11440 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:44.937475 11262 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:45.035620 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 77948us
I20260709 13:14:45.057400 11515 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:45.058933 12375 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:45.062970 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 43506us
W20260709 13:14:45.067339 11409 consensus_peers.cc:597] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 99667fcf16014dc1b9b174f0a37d16af. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:45.068370 11368 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:45.070145 12384 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:45.086391 12386 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.088196 11368 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.088394 11515 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.100118 11511 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.101930 11263 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:45.115536 11483 consensus_peers.cc:597] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6002d9cfb1c74b70b8ec301e31e4bbe9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:45.125597 11368 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:45.127615 12321 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:45.129843 11445 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:45.132093 12321 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:45.140569 12321 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.144359 11366 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.148444 11445 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.151959 11511 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.156931 11263 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.170881 11445 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:45.176316 11483 consensus_peers.cc:597] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf199b70fbb94d728a09194d18ca3b1e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:45.180938 12245 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:45.181456 11368 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:45.183884 12245 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:45.188447 12397 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.190017 11440 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.193287 11368 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:45.204074 11263 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:45.582765 12402 ts_tablet_manager.cc:933] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:45.631062 12404 ts_tablet_manager.cc:933] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:45.662386 12402 tablet_copy_client.cc:323] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:45.665536 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 7074ddeb0b7b4c1395b798352ff7db0a from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:45.666319 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 7074ddeb0b7b4c1395b798352ff7db0a from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-7074ddeb0b7b4c1395b798352ff7db0a
I20260709 13:14:45.678074 11380 tablet_copy_source_session.cc:215] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:45.682621 12402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7074ddeb0b7b4c1395b798352ff7db0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:45.681391 12404 tablet_copy_client.cc:323] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:14:45.695068 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 99667fcf16014dc1b9b174f0a37d16af from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:14:45.695778 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 99667fcf16014dc1b9b174f0a37d16af from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-99667fcf16014dc1b9b174f0a37d16af
I20260709 13:14:45.716729 11455 tablet_copy_source_session.cc:215] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:45.724588 12404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99667fcf16014dc1b9b174f0a37d16af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:45.739444 12402 tablet_copy_client.cc:806] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:45.740219 12402 tablet_copy_client.cc:670] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:45.754566 12402 tablet_copy_client.cc:538] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:45.756451 12407 ts_tablet_manager.cc:933] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:45.766546 12406 ts_tablet_manager.cc:933] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:45.766520 12407 tablet_copy_client.cc:323] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:45.774284 12406 tablet_copy_client.cc:323] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:45.774686 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet cf199b70fbb94d728a09194d18ca3b1e from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:45.774948 12404 tablet_copy_client.cc:806] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:45.770805 12402 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:45.775337 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet cf199b70fbb94d728a09194d18ca3b1e from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-cf199b70fbb94d728a09194d18ca3b1e
I20260709 13:14:45.776172 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:45.776713 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-6002d9cfb1c74b70b8ec301e31e4bbe9
I20260709 13:14:45.782661 12404 tablet_copy_client.cc:670] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:45.785163 11529 tablet_copy_source_session.cc:215] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:45.787364 11528 tablet_copy_source_session.cc:215] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:45.788796 12406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:45.802513 12404 tablet_copy_client.cc:538] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:45.803895 12407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf199b70fbb94d728a09194d18ca3b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:45.822150 12404 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:45.840734 12407 tablet_copy_client.cc:806] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:45.843896 12407 tablet_copy_client.cc:670] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:45.848171 12407 tablet_copy_client.cc:538] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:45.849648 12406 tablet_copy_client.cc:806] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:45.852108 12406 tablet_copy_client.cc:670] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:45.856232 12406 tablet_copy_client.cc:538] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:45.861438 12402 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:45.864023 12402 tablet_bootstrap.cc:492] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:45.864782 12402 ts_tablet_manager.cc:1403] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.094s	user 0.017s	sys 0.013s
I20260709 13:14:45.870973 12402 raft_consensus.cc:359] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.872365 12402 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:45.872664 12404 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:45.873642 12404 tablet_bootstrap.cc:492] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:45.873150 12402 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.879907 12407 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:45.865366 12406 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:45.877645 12404 ts_tablet_manager.cc:1403] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.056s	user 0.026s	sys 0.000s
I20260709 13:14:45.883519 12402 ts_tablet_manager.cc:1434] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.004s
I20260709 13:14:45.890614 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-7074ddeb0b7b4c1395b798352ff7db0a received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:45.891114 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-7074ddeb0b7b4c1395b798352ff7db0a on tablet 7074ddeb0b7b4c1395b798352ff7db0a with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:45.897557 12404 raft_consensus.cc:359] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.901636 12404 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:45.902475 12404 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.906039 12404 ts_tablet_manager.cc:1434] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:14:45.908238 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-99667fcf16014dc1b9b174f0a37d16af received from {username='slave'} at 127.0.0.1:41514
I20260709 13:14:45.908737 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-99667fcf16014dc1b9b174f0a37d16af on tablet 99667fcf16014dc1b9b174f0a37d16af with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:45.928082 12406 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:45.928954 12406 tablet_bootstrap.cc:492] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:45.929565 12406 ts_tablet_manager.cc:1403] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.064s	user 0.026s	sys 0.010s
I20260709 13:14:45.932446 12406 raft_consensus.cc:359] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.933105 12406 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:45.933604 12406 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.935472 12406 ts_tablet_manager.cc:1434] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:14:45.937685 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-6002d9cfb1c74b70b8ec301e31e4bbe9 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:45.938056 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-6002d9cfb1c74b70b8ec301e31e4bbe9 on tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:45.940333 12407 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:45.941296 12407 tablet_bootstrap.cc:492] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:45.942770 12407 ts_tablet_manager.cc:1403] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.009s
I20260709 13:14:45.945883 12407 raft_consensus.cc:359] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.946753 12407 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:45.947427 12407 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:45.949508 12407 ts_tablet_manager.cc:1434] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:14:45.952431 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-cf199b70fbb94d728a09194d18ca3b1e received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:45.952924 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-cf199b70fbb94d728a09194d18ca3b1e on tablet cf199b70fbb94d728a09194d18ca3b1e with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:45.989018 12067 raft_consensus.cc:1217] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:46.193410 12067 raft_consensus.cc:1217] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:46.280186 12067 raft_consensus.cc:1217] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:46.321265 12067 raft_consensus.cc:1217] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:46.485525 12238 raft_consensus.cc:1064] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:46.487627 12238 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:46.496137 12067 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.497612 12370 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:46.502372 11511 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.502404 11445 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.503835 12370 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.506047 12401 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.549909 12370 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.555822 12067 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.575541 11263 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.562772 11445 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.597693 11517 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.625144 11368 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:46.630785 12067 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:46.631760 11511 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:46.632274 12377 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:46.634017 12370 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:46.641134 12401 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.642400 11511 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.645761 12415 raft_consensus.cc:1064] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:46.646950 12067 raft_consensus.cc:2955] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.648190 12415 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:46.659549 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7074ddeb0b7b4c1395b798352ff7db0a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:46.676991 11266 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.679388 12067 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.679857 11515 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.681615 11368 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.684190 12304 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.685830 12304 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.688449 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 7074ddeb0b7b4c1395b798352ff7db0a with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:46.692376 12415 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:46.713637 12427 tablet_replica.cc:333] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:46.712898 12387 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.715274 12427 raft_consensus.cc:2243] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:46.716044 12427 raft_consensus.cc:2272] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:46.720301 12427 ts_tablet_manager.cc:1916] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:46.739029 11368 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.749105 12427 ts_tablet_manager.cc:1929] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:46.749699 12427 log.cc:1199] T 7074ddeb0b7b4c1395b798352ff7db0a P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/7074ddeb0b7b4c1395b798352ff7db0a
I20260709 13:14:46.750465 11266 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:46.752251 12069 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.753470 12374 raft_consensus.cc:1064] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:46.756143 12374 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:46.759864 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 7074ddeb0b7b4c1395b798352ff7db0a (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:46.762888 11511 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.766569 12067 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.769312 11366 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.770620 11445 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.771319 12420 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:46.774519 12245 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.779291 12397 raft_consensus.cc:1064] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:46.781840 12397 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:46.791035 12374 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:46.807523 11442 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:46.820134 12069 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:46.823335 12415 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:46.836927 11364 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:46.831828 12415 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.864041 12069 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.863173 12387 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:14:46.890741 11363 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.892180 12374 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.896603 11437 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.899149 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 99667fcf16014dc1b9b174f0a37d16af with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:46.897317 12069 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:46.904600 12374 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.906967 12420 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:46.920461 11366 raft_consensus.cc:2955] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.922109 11443 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.923202 12245 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.923802 12067 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.932495 11266 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.950064 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 99667fcf16014dc1b9b174f0a37d16af with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:46.952232 12440 tablet_replica.cc:333] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:46.953845 12440 raft_consensus.cc:2243] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:46.954783 12440 raft_consensus.cc:2272] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:46.955976 11263 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.960923 12440 ts_tablet_manager.cc:1916] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:46.962649 12420 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.965629 12069 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:46.977988 12440 ts_tablet_manager.cc:1929] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:46.978502 12440 log.cc:1199] T 99667fcf16014dc1b9b174f0a37d16af P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/99667fcf16014dc1b9b174f0a37d16af
I20260709 13:14:46.983650 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 99667fcf16014dc1b9b174f0a37d16af (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:46.986754 11366 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:14:46.987084 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49029.3us
I20260709 13:14:46.993759 11262 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.994416 11368 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.003556 11437 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.033327 11517 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:47.034114 11511 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:47.038036 12069 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:47.039047 11437 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:47.039830 12397 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:47.041164 12438 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:47.050132 11368 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:47.050503 12245 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.052805 12397 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:47.053529 11437 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.055106 12069 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:47.058450 12438 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:47.060148 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cf199b70fbb94d728a09194d18ca3b1e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:47.060840 12069 raft_consensus.cc:2955] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.068149 12374 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.072011 12067 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.082293 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:47.070389 11368 raft_consensus.cc:2955] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.071342 11267 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:47.114759 11262 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:47.121999 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet cf199b70fbb94d728a09194d18ca3b1e with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:47.133376 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:47.134240 12427 tablet_replica.cc:333] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:47.134369 12441 tablet_replica.cc:333] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:47.135231 12427 raft_consensus.cc:2243] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:47.136075 12441 raft_consensus.cc:2243] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:47.136089 12427 raft_consensus.cc:2272] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:47.137423 12441 raft_consensus.cc:2272] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:47.139683 12427 ts_tablet_manager.cc:1916] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:47.141680 12441 ts_tablet_manager.cc:1916] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:47.153450 12427 ts_tablet_manager.cc:1929] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:47.153988 12427 log.cc:1199] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/6002d9cfb1c74b70b8ec301e31e4bbe9
I20260709 13:14:47.155596 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 6002d9cfb1c74b70b8ec301e31e4bbe9 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:47.156813 12441 ts_tablet_manager.cc:1929] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:47.157303 12441 log.cc:1199] T cf199b70fbb94d728a09194d18ca3b1e P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/cf199b70fbb94d728a09194d18ca3b1e
I20260709 13:14:47.158994 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet cf199b70fbb94d728a09194d18ca3b1e (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:48.207988 11366 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.214409 11511 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:48.214787 11442 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:48.216295 11334 consensus_peers.cc:597] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81760accce3a440a82323bd3ff53c551. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:48.216794 12443 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.217927 12377 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.227648 11442 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.227458 11511 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.225862 12401 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.244650 11262 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.253160 11517 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.260236 11442 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:48.260556 11483 consensus_peers.cc:597] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 063871012da74857a239cdf01c76e5ab. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:48.261591 12374 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.262826 11366 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:48.264482 12374 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.278383 12374 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.279942 11443 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.292035 11366 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.294160 11442 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.334918 11517 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:48.335019 11409 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98fc3c2677604f3bb787b67c5e5b6012. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:48.336457 12435 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.339843 11366 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:48.344314 12387 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.340813 11266 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.362124 12387 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.364459 11363 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.365144 11517 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.377821 11442 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.383304 11266 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.387738 11511 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:48.390877 12459 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:14:48.392838 11409 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bce024e6afd54cf0992c0af40df6a4eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:48.398505 11363 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:48.401103 12459 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:48.409675 12387 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.411523 11366 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.435860 11262 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.450941 11511 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:48.688776 12463 ts_tablet_manager.cc:933] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:48.690639 12463 tablet_copy_client.cc:323] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:48.692400 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 81760accce3a440a82323bd3ff53c551 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:48.692911 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 81760accce3a440a82323bd3ff53c551 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-81760accce3a440a82323bd3ff53c551
I20260709 13:14:48.714128 11380 tablet_copy_source_session.cc:215] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:48.718981 12463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81760accce3a440a82323bd3ff53c551. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:48.735749 12465 ts_tablet_manager.cc:933] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:48.738171 12463 tablet_copy_client.cc:806] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:48.738024 12465 tablet_copy_client.cc:323] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:48.738715 12463 tablet_copy_client.cc:670] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:48.739971 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 063871012da74857a239cdf01c76e5ab from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:48.740486 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 063871012da74857a239cdf01c76e5ab from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-063871012da74857a239cdf01c76e5ab
I20260709 13:14:48.749480 11529 tablet_copy_source_session.cc:215] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:48.751016 12463 tablet_copy_client.cc:538] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:48.770177 12465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063871012da74857a239cdf01c76e5ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:48.787384 12463 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:48.789772 12467 ts_tablet_manager.cc:933] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:48.798272 12465 tablet_copy_client.cc:806] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:48.798923 12465 tablet_copy_client.cc:670] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:48.802337 12467 tablet_copy_client.cc:323] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:14:48.803349 12465 tablet_copy_client.cc:538] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:48.804826 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 98fc3c2677604f3bb787b67c5e5b6012 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:14:48.805344 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 98fc3c2677604f3bb787b67c5e5b6012 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-98fc3c2677604f3bb787b67c5e5b6012
I20260709 13:14:48.815172 12465 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:48.817296 11455 tablet_copy_source_session.cc:215] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:48.821847 12463 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:48.822597 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98fc3c2677604f3bb787b67c5e5b6012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:48.822845 12463 tablet_bootstrap.cc:492] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:48.823583 12463 ts_tablet_manager.cc:1403] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.013s
I20260709 13:14:48.833655 12463 raft_consensus.cc:359] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.834584 12463 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:48.835778 12463 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.846063 12463 ts_tablet_manager.cc:1434] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.006s
I20260709 13:14:48.849865 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-81760accce3a440a82323bd3ff53c551 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:48.850288 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-81760accce3a440a82323bd3ff53c551 on tablet 81760accce3a440a82323bd3ff53c551 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:48.867169 12467 tablet_copy_client.cc:806] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:48.868589 12467 tablet_copy_client.cc:670] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:48.872427 12467 tablet_copy_client.cc:538] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:48.875339 12465 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:48.876402 12465 tablet_bootstrap.cc:492] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:48.877128 12465 ts_tablet_manager.cc:1403] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.062s	user 0.035s	sys 0.019s
I20260709 13:14:48.880275 12465 raft_consensus.cc:359] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.881181 12465 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:48.881867 12465 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.884554 12465 ts_tablet_manager.cc:1434] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:14:48.886299 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-063871012da74857a239cdf01c76e5ab received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:48.886691 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-063871012da74857a239cdf01c76e5ab on tablet 063871012da74857a239cdf01c76e5ab with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:48.888726 12467 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:48.900874 12465 ts_tablet_manager.cc:933] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:48.906900 12465 tablet_copy_client.cc:323] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:14:48.908542 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet bce024e6afd54cf0992c0af40df6a4eb from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:14:48.909050 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet bce024e6afd54cf0992c0af40df6a4eb from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-bce024e6afd54cf0992c0af40df6a4eb
I20260709 13:14:48.916357 11455 tablet_copy_source_session.cc:215] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:48.917424 12467 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:48.918162 12467 tablet_bootstrap.cc:492] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:48.918682 12467 ts_tablet_manager.cc:1403] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.000s
I20260709 13:14:48.919553 12465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce024e6afd54cf0992c0af40df6a4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:48.920898 12467 raft_consensus.cc:359] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.922749 12467 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:48.923432 12467 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:48.934746 12467 ts_tablet_manager.cc:1434] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:14:48.942142 12465 tablet_copy_client.cc:806] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:48.942960 12465 tablet_copy_client.cc:670] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:48.946561 12465 tablet_copy_client.cc:538] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:48.956197 12465 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:48.981042 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-98fc3c2677604f3bb787b67c5e5b6012 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:14:48.981583 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-98fc3c2677604f3bb787b67c5e5b6012 on tablet 98fc3c2677604f3bb787b67c5e5b6012 with peer b1906ab37ba541f493936685a808c8a9
W20260709 13:14:48.991473 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47544.2us
I20260709 13:14:48.999715 12465 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:49.000738 12465 tablet_bootstrap.cc:492] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:49.001453 12465 ts_tablet_manager.cc:1403] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.007s
I20260709 13:14:49.004503 12465 raft_consensus.cc:359] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:49.005311 12465 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:49.005924 12465 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:49.008210 12465 ts_tablet_manager.cc:1434] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:14:49.009681 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-bce024e6afd54cf0992c0af40df6a4eb received from {username='slave'} at 127.0.0.1:41514
I20260709 13:14:49.010073 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-bce024e6afd54cf0992c0af40df6a4eb on tablet bce024e6afd54cf0992c0af40df6a4eb with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:49.266930 12069 raft_consensus.cc:1217] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:49.284569 12069 raft_consensus.cc:1217] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:49.295704 12069 raft_consensus.cc:1217] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:49.429100 12069 raft_consensus.cc:1217] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:49.692605 12474 raft_consensus.cc:1064] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:49.694975 12474 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:49.701568 12069 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.701833 11517 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.703243 12387 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.704737 11366 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.705792 12474 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.707675 12387 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.744295 12401 raft_consensus.cc:1064] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:49.747119 12401 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:49.748497 12435 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.755971 12069 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.772967 11262 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.779644 12397 raft_consensus.cc:1064] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:49.781843 12397 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:49.808020 11366 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.822849 12474 raft_consensus.cc:1064] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:49.825377 12474 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:49.826416 12067 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.826296 11517 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.834946 12453 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.838814 12067 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.840610 11511 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.842643 11363 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.843804 12447 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.852253 12485 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.854830 12447 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:49.858039 11443 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.874172 12484 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.888337 12494 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.906981 11511 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.914935 11437 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.919673 11515 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.920956 12067 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.923169 12435 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.924084 11366 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:49.924865 12474 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:49.930018 12474 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:49.925490 11263 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.931783 12485 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.930007 12067 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.939879 12069 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.949940 12475 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:49.951668 11437 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.964265 12387 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.973215 11366 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.973706 11445 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.984062 11368 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:49.987709 12069 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.002398 11515 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.055202 11437 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:50.055748 11263 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.059151 11263 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.064719 12069 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.066875 12387 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:50.073370 12435 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.086711 12068 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.088871 11515 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.098438 12387 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:50.102921 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 98fc3c2677604f3bb787b67c5e5b6012 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:50.104827 11515 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:50.112541 11363 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.112543 12069 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.113962 12454 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:50.115269 12485 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:50.123979 11263 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.130117 11515 raft_consensus.cc:2955] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.135061 12484 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.143076 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 98fc3c2677604f3bb787b67c5e5b6012 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:50.148631 11363 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.153508 11437 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:50.157375 12068 raft_consensus.cc:2955] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.158332 12504 tablet_replica.cc:333] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:50.159832 12504 raft_consensus.cc:2243] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:50.160602 12504 raft_consensus.cc:2272] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:50.161969 11366 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.164306 12504 ts_tablet_manager.cc:1916] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:50.164837 12474 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:50.175038 12387 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.179447 12504 ts_tablet_manager.cc:1929] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:50.179955 12504 log.cc:1199] T 98fc3c2677604f3bb787b67c5e5b6012 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/98fc3c2677604f3bb787b67c5e5b6012
I20260709 13:14:50.181583 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 98fc3c2677604f3bb787b67c5e5b6012 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:50.180915 11368 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.183688 12067 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.185009 12435 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:50.189726 12067 raft_consensus.cc:2955] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.193689 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bce024e6afd54cf0992c0af40df6a4eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:50.199550 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 063871012da74857a239cdf01c76e5ab with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:50.211284 11267 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.219789 11263 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.225437 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet bce024e6afd54cf0992c0af40df6a4eb with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:50.229969 12505 tablet_replica.cc:333] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:50.231026 12505 raft_consensus.cc:2243] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:50.231760 12505 raft_consensus.cc:2272] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:50.236125 12505 ts_tablet_manager.cc:1916] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:50.239090 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 063871012da74857a239cdf01c76e5ab with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:50.272460 12506 tablet_replica.cc:333] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:50.278627 12506 raft_consensus.cc:2243] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:50.279533 12506 raft_consensus.cc:2272] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:50.279956 12505 ts_tablet_manager.cc:1929] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:50.280368 12505 log.cc:1199] T bce024e6afd54cf0992c0af40df6a4eb P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/bce024e6afd54cf0992c0af40df6a4eb
I20260709 13:14:50.282037 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet bce024e6afd54cf0992c0af40df6a4eb (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:50.284021 12506 ts_tablet_manager.cc:1916] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:50.299329 12506 ts_tablet_manager.cc:1929] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:50.299737 12506 log.cc:1199] T 063871012da74857a239cdf01c76e5ab P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/063871012da74857a239cdf01c76e5ab
I20260709 13:14:50.301158 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 063871012da74857a239cdf01c76e5ab (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:50.447593 11363 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:50.452723 12068 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.452723 11437 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:50.454245 12491 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:50.455240 12492 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:50.461200 12475 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.463351 12067 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.464234 11437 raft_consensus.cc:2955] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.471102 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81760accce3a440a82323bd3ff53c551 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:50.478135 11263 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:50.489022 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 81760accce3a440a82323bd3ff53c551 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:50.490069 12505 tablet_replica.cc:333] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:50.490880 12505 raft_consensus.cc:2243] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:50.491596 12505 raft_consensus.cc:2272] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:50.495440 12505 ts_tablet_manager.cc:1916] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:50.510372 12505 ts_tablet_manager.cc:1929] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:50.510658 12505 log.cc:1199] T 81760accce3a440a82323bd3ff53c551 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/81760accce3a440a82323bd3ff53c551
I20260709 13:14:50.511744 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 81760accce3a440a82323bd3ff53c551 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:51.174686 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:51.231565 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:51.232229 11368 raft_consensus.cc:606] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:14:51.234225 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:51.234588 11368 raft_consensus.cc:606] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:14:51.236369 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d475300e0054608a5d243d859d6609b"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:51.236716 11368 raft_consensus.cc:606] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:51.238610 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:51.238958 11368 raft_consensus.cc:606] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:51.251353 11515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:51.251798 11515 raft_consensus.cc:606] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:51.252712 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:51.253715 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:51.317869 12453 raft_consensus.cc:993] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:51.318535 12507 raft_consensus.cc:1081] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:51.319782 11368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:51.320330 11368 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:51.320647 11368 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.324821 11368 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.326417 11368 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:51.327023 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:51.327337 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:51.327678 11443 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.327946 11515 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:51.328312 11515 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:51.328938 11515 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.329876 11515 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.332557 11443 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:51.333662 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:51.334369 12489 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:51.334741 11515 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:51.335829 12489 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:51.336706 12489 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.347108 11267 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.360213 12512 raft_consensus.cc:993] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:51.360602 12489 raft_consensus.cc:1081] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:51.362025 11443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:14:51.362488 11443 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:51.362742 11443 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.367719 11443 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.369095 11443 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:51.369639 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:51.370016 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:51.370157 11368 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:51.370453 11368 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:51.370468 11515 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.371134 11368 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.371968 11368 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.374650 11515 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:51.375666 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:51.376051 11368 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:51.376315 12474 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:51.377555 12474 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:51.378396 12474 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.387936 11267 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.394701 12512 raft_consensus.cc:993] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:51.395155 12489 raft_consensus.cc:1081] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:51.396520 12067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54370
I20260709 13:14:51.396929 12067 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:51.397171 12067 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.402099 12067 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:51.403985 12067 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:51.404553 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:51.404865 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:51.405124 11368 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:51.405395 11515 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.405437 11368 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:51.406234 11368 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:51.407361 11368 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.411808 11515 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:51.412994 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:51.413300 12512 raft_consensus.cc:993] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:51.413738 12512 raft_consensus.cc:1081] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:51.414286 11368 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:51.415126 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d475300e0054608a5d243d859d6609b"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54370
I20260709 13:14:51.415624 12068 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:51.415868 12068 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.424921 12519 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:51.424842 12068 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:51.427295 12519 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:51.428112 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:51.428390 12068 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:51.428411 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:51.428704 11368 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:51.428588 12519 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:51.429301 11443 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.429333 11368 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:51.430380 11368 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:51.431559 11368 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.434969 11443 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:51.436049 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:51.436062 11368 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:51.438807 11267 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:51.440697 12519 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:51.441243 12519 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:51.441956 12519 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:51.451947 11267 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:51.545161 11515 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:51.551200 11443 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:51.551836 11368 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:51.551993 11483 consensus_peers.cc:597] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcf62f83a1754dcc80291daa521a9c06. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.553072 12508 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.553900 12453 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.561205 12453 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.563509 11443 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.571796 11368 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.580366 11437 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:51.579046 11267 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
W20260709 13:14:51.596539 11409 consensus_peers.cc:597] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38ecdb2d3e8745889c0343749502c9f5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.597213 11368 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:51.600055 11515 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:51.604449 12517 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.607999 12474 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.622848 12517 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.625264 11368 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.640232 12512 raft_consensus.cc:993] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:51.639881 11515 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.640692 12512 raft_consensus.cc:1081] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:51.642294 11443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:14:51.641955 11517 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:51.642787 11443 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:51.643163 11443 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.647503 11443 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.647078 11263 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.655997 11443 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:51.657979 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:51.658636 11368 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:51.658954 11368 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:51.660573 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:51.661127 11511 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.668952 11511 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:51.669735 11515 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:51.670969 11443 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:51.659783 11368 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.673089 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:51.674182 12517 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:51.674715 12517 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:51.672335 12507 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:14:51.675334 12517 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:51.686939 11267 catalog_manager.cc:5697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.692691 11363 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:51.693089 11483 consensus_peers.cc:597] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7dab116931a64024ac61a35a6d29a836. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.701148 12453 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:51.672343 12489 consensus_queue.cc:1175] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34a42aa6aa0a4778a532c6433d70b981" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:51.704402 11368 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:51.704986 12453 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.711167 11366 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.713361 11437 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.719798 11363 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:51.726660 11267 catalog_manager.cc:5697] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.710448 11368 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
W20260709 13:14:51.747068 11334 consensus_peers.cc:597] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a7effda9a8a1409cbe25d55ed7cf2964. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.749620 11511 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:51.750785 11437 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:51.752545 12489 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.753446 12401 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.760360 12401 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.761806 11515 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.767827 11437 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.785698 11263 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:51.809327 11515 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:51.810364 12517 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.830967 11363 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:51.832567 12474 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.851261 11515 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:51.855014 12538 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.880939 11437 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:51.883061 12489 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:51.905544 11368 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:51.907196 12520 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:51.907474 11363 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:51.909071 12520 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:51.937150 11437 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:51.940021 12519 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:51.991434 11515 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:52.037520 12553 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:52.045054 12556 ts_tablet_manager.cc:933] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:52.055655 12556 tablet_copy_client.cc:323] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:14:52.069095 11515 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:52.070235 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 38ecdb2d3e8745889c0343749502c9f5 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:14:52.070834 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 38ecdb2d3e8745889c0343749502c9f5 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-38ecdb2d3e8745889c0343749502c9f5
I20260709 13:14:52.075250 12517 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:52.079909 11455 tablet_copy_source_session.cc:215] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.084256 12556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ecdb2d3e8745889c0343749502c9f5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:52.119066 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 59039.7us
I20260709 13:14:52.144436 12556 tablet_copy_client.cc:806] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.145625 12556 tablet_copy_client.cc:670] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.152650 12562 ts_tablet_manager.cc:933] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:52.162611 11368 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:52.169869 12474 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:52.172504 12562 tablet_copy_client.cc:323] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:52.174765 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet a7effda9a8a1409cbe25d55ed7cf2964 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:14:52.175384 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet a7effda9a8a1409cbe25d55ed7cf2964 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-a7effda9a8a1409cbe25d55ed7cf2964
I20260709 13:14:52.185683 11380 tablet_copy_source_session.cc:215] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.188988 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7effda9a8a1409cbe25d55ed7cf2964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:52.191679 12556 tablet_copy_client.cc:538] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.205304 12556 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:52.207559 12562 tablet_copy_client.cc:806] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.208362 12562 tablet_copy_client.cc:670] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.212550 12562 tablet_copy_client.cc:538] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.231381 12566 ts_tablet_manager.cc:933] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:52.233219 12562 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:52.234937 12565 ts_tablet_manager.cc:933] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:52.249814 12566 tablet_copy_client.cc:323] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:52.252306 12565 tablet_copy_client.cc:323] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:52.253336 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 7dab116931a64024ac61a35a6d29a836 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:52.253886 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 7dab116931a64024ac61a35a6d29a836 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-7dab116931a64024ac61a35a6d29a836
I20260709 13:14:52.254375 11528 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet bcf62f83a1754dcc80291daa521a9c06 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:52.254896 11528 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet bcf62f83a1754dcc80291daa521a9c06 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-bcf62f83a1754dcc80291daa521a9c06
I20260709 13:14:52.256657 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:52.262339 11528 tablet_copy_source_session.cc:215] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.269135 12556 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.269579 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf62f83a1754dcc80291daa521a9c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:52.270136 12556 tablet_bootstrap.cc:492] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:52.270901 12556 ts_tablet_manager.cc:1403] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.010s
I20260709 13:14:52.272637 11529 tablet_copy_source_session.cc:215] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.273864 12556 raft_consensus.cc:359] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.274695 12556 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:52.275301 12556 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.282187 12556 ts_tablet_manager.cc:1434] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:14:52.287037 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-38ecdb2d3e8745889c0343749502c9f5 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:14:52.287577 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-38ecdb2d3e8745889c0343749502c9f5 on tablet 38ecdb2d3e8745889c0343749502c9f5 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.288266 12566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dab116931a64024ac61a35a6d29a836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:52.369107 12565 tablet_copy_client.cc:806] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.369756 12565 tablet_copy_client.cc:670] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.369992 12562 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.371109 12562 tablet_bootstrap.cc:492] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:52.374516 12565 tablet_copy_client.cc:538] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.410717 12566 tablet_copy_client.cc:806] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.411787 12566 tablet_copy_client.cc:670] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.416779 12566 tablet_copy_client.cc:538] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.421058 12562 ts_tablet_manager.cc:1403] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.188s	user 0.036s	sys 0.042s
I20260709 13:14:52.421396 12565 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:52.424326 12562 raft_consensus.cc:359] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.425132 12562 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:52.425777 12562 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.428157 12562 ts_tablet_manager.cc:1434] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:14:52.430768 12566 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:52.437389 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-a7effda9a8a1409cbe25d55ed7cf2964 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:52.437963 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-a7effda9a8a1409cbe25d55ed7cf2964 on tablet a7effda9a8a1409cbe25d55ed7cf2964 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.466163 12565 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.467156 12565 tablet_bootstrap.cc:492] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:52.467903 12565 ts_tablet_manager.cc:1403] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.047s	user 0.023s	sys 0.005s
I20260709 13:14:52.468533 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:52.471660 12565 raft_consensus.cc:359] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.472602 12565 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:52.473284 12565 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.476269 12566 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.476583 12565 ts_tablet_manager.cc:1434] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:14:52.480377 12068 raft_consensus.cc:1217] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:52.482613 12566 tablet_bootstrap.cc:492] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:52.483497 12566 ts_tablet_manager.cc:1403] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.014s
I20260709 13:14:52.484000 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-bcf62f83a1754dcc80291daa521a9c06 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:52.484436 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-bcf62f83a1754dcc80291daa521a9c06 on tablet bcf62f83a1754dcc80291daa521a9c06 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.489559 12566 raft_consensus.cc:359] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.492007 12566 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:52.492867 12566 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:52.495550 12566 ts_tablet_manager.cc:1434] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:14:52.498184 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-7dab116931a64024ac61a35a6d29a836 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:52.498590 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-7dab116931a64024ac61a35a6d29a836 on tablet 7dab116931a64024ac61a35a6d29a836 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.553392 12068 raft_consensus.cc:1217] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:52.579811 11445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:14:52.580416 11445 raft_consensus.cc:606] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:52.584499 11517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:52.585052 11517 raft_consensus.cc:606] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:52.590718 11515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b9049e00f94cc1912970768037cf24"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:52.591277 11515 raft_consensus.cc:606] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.593864 11517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:52.594411 11517 raft_consensus.cc:606] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.596760 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:52.597155 11368 raft_consensus.cc:606] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.599300 11517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:52.599826 11517 raft_consensus.cc:606] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:52.600983 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:52.605057 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:52.697631 12453 raft_consensus.cc:993] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.698119 12559 raft_consensus.cc:1081] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.699468 12069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:14:52.700008 12069 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.700327 12069 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.707042 12069 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.709319 12069 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:52.709612 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:52.710184 11445 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.711833 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:52.712437 11517 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:52.712785 11517 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:52.713500 11517 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.715634 11517 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.715956 11445 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.717146 12034 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:52.718250 12553 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:52.718703 12553 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:52.721199 12553 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.726547 12512 raft_consensus.cc:993] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.726997 12512 raft_consensus.cc:1081] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.728809 12069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54370
I20260709 13:14:52.729386 12069 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.729754 12069 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.736099 12069 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.728601 11517 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.740324 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:52.740891 11366 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:52.741178 11366 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:52.741278 12069 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:52.741961 11366 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.743036 11366 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.744751 11263 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.748713 12068 raft_consensus.cc:1217] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:52.752276 11366 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.753722 12033 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:52.754519 12553 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:52.754967 12553 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:52.755791 12553 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.757985 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:52.758528 11517 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.759500 12508 raft_consensus.cc:993] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.759958 12453 raft_consensus.cc:1081] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.762059 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:14:52.762563 12068 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.762912 12068 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.769212 11517 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.769620 12068 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.772696 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:52.773182 11445 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.776175 12068 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:52.777024 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:52.777714 11517 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:52.778069 11517 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:52.778764 11517 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.779883 11445 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.780977 12034 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:52.781132 11517 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.781742 12551 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:52.782199 12551 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:52.782974 12551 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.787518 11517 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.791004 12069 raft_consensus.cc:1217] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:52.795287 12577 raft_consensus.cc:993] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.796478 12508 raft_consensus.cc:1081] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:52.800920 12067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78b9049e00f94cc1912970768037cf24"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:14:52.801518 12067 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.801843 12067 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.817716 12067 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.824542 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:52.824916 12067 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:52.825032 11368 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.825984 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:52.826509 11517 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:52.826771 11517 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:52.827627 11517 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.828606 11517 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.830904 11368 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.832427 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:52.833886 12553 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:52.834329 12553 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:52.834641 11517 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:52.835112 12553 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:52.840166 11263 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.841325 11263 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.859377 11263 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.869748 12571 raft_consensus.cc:993] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:52.870221 12571 raft_consensus.cc:1081] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:52.871800 11368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:14:52.872373 11368 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:52.872700 11368 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.880489 11368 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:52.882853 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:52.883366 11517 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.883462 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:52.884011 11437 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:52.884270 11437 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:52.885003 11437 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:52.885955 11437 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:52.887538 11368 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:52.890635 11517 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:52.891996 11333 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:52.892824 12512 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:52.893939 12512 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:52.894840 12512 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:52.896377 11437 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:52.905802 11263 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.022724 12539 raft_consensus.cc:1064] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:53.025380 12539 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:53.033311 12067 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.033771 11517 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.034718 11437 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.034773 12489 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:53.036538 12539 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:53.037690 12512 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:53.047371 12527 raft_consensus.cc:1064] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
W20260709 13:14:53.066144 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.137802 12527 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:14:53.140202 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 79292.9us
I20260709 13:14:53.159631 12559 raft_consensus.cc:993] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:53.160092 12559 raft_consensus.cc:1081] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:53.166271 12512 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.173584 12067 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.174069 11517 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.175143 12571 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:53.177254 11517 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.177012 12474 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:53.179903 11366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:53.185070 11366 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:53.185493 11366 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:53.183023 11368 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.198213 11368 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.198499 12589 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:53.199620 12553 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:53.207558 11366 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:53.210457 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:53.210999 11515 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:53.211263 11515 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:53.211967 11515 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:53.212807 11515 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:53.226176 11515 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:53.227277 11333 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:53.228363 12539 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:53.229131 12539 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:53.230106 12539 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:53.235492 11437 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.243366 11368 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.244848 11445 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.247231 12553 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:53.247520 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:53.248226 11442 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:53.255213 11442 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:53.270004 11267 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.262699 12589 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.285977 11515 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.286005 11519 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.287796 12553 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:53.289645 12551 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:53.290625 11517 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.299511 12553 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:53.300060 12067 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.311491 11364 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.287806 11366 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:53.367151 11515 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.367067 12603 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:14:53.373786 12603 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:53.374852 11442 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:53.389000 11437 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:14:53.389318 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 131613us
I20260709 13:14:53.393617 12595 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:53.397135 12604 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:53.409140 11511 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.422600 12067 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.436210 11516 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:53.438843 12595 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:53.451949 11263 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.456804 11266 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:53.481675 11443 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:53.493804 12067 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.495576 12571 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:53.500229 11364 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:53.508028 12068 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.508680 11516 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.510504 12600 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:53.512609 12600 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:14:53.515277 12517 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.519667 12539 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.521514 11519 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.524349 11364 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.524446 12067 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.525976 12571 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:53.526608 12068 raft_consensus.cc:2955] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.538977 11364 raft_consensus.cc:2955] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.543021 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a7effda9a8a1409cbe25d55ed7cf2964 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:53.543037 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 38ecdb2d3e8745889c0343749502c9f5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:53.563001 11266 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.566298 12577 raft_consensus.cc:1064] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:53.570206 12577 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:53.575816 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 38ecdb2d3e8745889c0343749502c9f5 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:53.583108 11262 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.595626 12619 tablet_replica.cc:333] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:53.597527 12619 raft_consensus.cc:2243] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:53.598330 12619 raft_consensus.cc:2272] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:53.601879 12619 ts_tablet_manager.cc:1916] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:53.606345 11443 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.606823 12559 raft_consensus.cc:1064] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:53.607919 12615 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:53.609062 12559 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:53.614993 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet a7effda9a8a1409cbe25d55ed7cf2964 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:53.621871 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:53.630439 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 48999.5us
I20260709 13:14:53.652134 11445 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.654446 12625 tablet_replica.cc:333] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:53.653888 12453 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:14:53.656168 12068 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.659173 11364 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.664850 12619 ts_tablet_manager.cc:1929] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:53.666132 12068 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.668689 12619 log.cc:1199] T 38ecdb2d3e8745889c0343749502c9f5 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/38ecdb2d3e8745889c0343749502c9f5
I20260709 13:14:53.670964 12625 raft_consensus.cc:2243] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:53.671500 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 38ecdb2d3e8745889c0343749502c9f5 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:53.671805 12625 raft_consensus.cc:2272] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:53.675716 11364 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:53.681645 12453 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:53.683432 11516 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:53.685535 12453 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:53.685571 12600 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:53.688239 12625 ts_tablet_manager.cc:1916] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:53.710606 12625 ts_tablet_manager.cc:1929] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:53.718938 12453 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:53.711073 12625 log.cc:1199] T a7effda9a8a1409cbe25d55ed7cf2964 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/a7effda9a8a1409cbe25d55ed7cf2964
I20260709 13:14:53.721071 11445 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:53.722844 12489 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:53.723137 12620 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:53.723884 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet a7effda9a8a1409cbe25d55ed7cf2964 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:53.735855 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:53.767539 12559 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.772074 12577 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.776752 11366 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.776846 11364 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.786970 11445 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.794217 12067 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.801839 12069 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.810253 11267 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.821050 11443 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.838285 11266 catalog_manager.cc:5697] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.868935 11516 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:53.874315 11519 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:53.884306 12068 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.886140 12559 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:53.886615 11443 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.895519 12577 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:53.898294 12577 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.909487 11519 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "063871012da74857a239cdf01c76e5ab"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:53.916196 11519 raft_consensus.cc:606] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:53.909000 12068 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.915948 11443 raft_consensus.cc:2955] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.924377 12068 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.926075 12621 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:53.931361 11364 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:53.930960 11262 catalog_manager.cc:5697] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.941686 12620 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:53.941294 12508 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.944012 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7dab116931a64024ac61a35a6d29a836 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:53.952025 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 7dab116931a64024ac61a35a6d29a836 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:14:53.952632 12069 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.959249 11366 raft_consensus.cc:2955] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:53.970952 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bcf62f83a1754dcc80291daa521a9c06 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:53.973083 12633 tablet_replica.cc:333] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:14:53.973894 12633 raft_consensus.cc:2243] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:53.974466 12633 raft_consensus.cc:2272] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:53.978682 12633 ts_tablet_manager.cc:1916] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:53.979151 11364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:53.989320 11364 raft_consensus.cc:606] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:14:53.993494 11364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:53.994066 11364 raft_consensus.cc:606] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:53.997602 11262 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:54.000186 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:54.000736 11516 raft_consensus.cc:606] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:54.003242 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:54.003818 11516 raft_consensus.cc:606] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:54.009038 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:54.010535 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:54.012434 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet bcf62f83a1754dcc80291daa521a9c06 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:54.013341 12625 tablet_replica.cc:333] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:54.014308 12625 raft_consensus.cc:2243] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:54.014896 12625 raft_consensus.cc:2272] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:54.017809 12625 ts_tablet_manager.cc:1916] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:54.021534 12633 ts_tablet_manager.cc:1929] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:54.021950 12633 log.cc:1199] T 7dab116931a64024ac61a35a6d29a836 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/7dab116931a64024ac61a35a6d29a836
I20260709 13:14:54.023521 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 7dab116931a64024ac61a35a6d29a836 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:54.035828 12625 ts_tablet_manager.cc:1929] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:54.036257 12625 log.cc:1199] T bcf62f83a1754dcc80291daa521a9c06 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/bcf62f83a1754dcc80291daa521a9c06
I20260709 13:14:54.037501 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet bcf62f83a1754dcc80291daa521a9c06 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:54.145845 12539 raft_consensus.cc:993] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:54.146251 12598 raft_consensus.cc:1081] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:54.147439 11445 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:14:54.147835 11445 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.148082 11445 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.152177 11445 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.153671 11445 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:54.154376 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:54.154598 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:54.154944 11364 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:54.155110 11516 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.155230 11364 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:54.155900 11364 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.156677 11364 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.160815 11516 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:54.161206 11364 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:54.161804 11408 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:54.162837 12517 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:54.163208 12517 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:54.163944 12517 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.173234 11263 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:54.286312 12508 raft_consensus.cc:993] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:54.286722 12559 raft_consensus.cc:1081] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:54.288117 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "063871012da74857a239cdf01c76e5ab"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:14:54.288551 12068 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.288800 12068 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.292928 12068 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:54.294397 12068 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:54.295364 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:54.295518 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:54.296114 11364 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.296218 11516 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:54.296619 11516 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:54.297336 11516 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:54.298303 11516 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.302191 11364 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:54.303424 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:54.304337 12553 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:54.304859 12553 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:54.305751 12553 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:54.306999 11516 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:54.315820 11263 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:54.352206 12559 raft_consensus.cc:993] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:54.352612 12508 raft_consensus.cc:1081] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:54.353817 11364 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:54.354215 11364 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.354429 11364 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.358564 11364 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.359928 11364 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:54.360574 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:54.360909 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:54.361285 11445 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.361730 11516 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:54.362047 11516 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:54.363581 11516 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.366255 11516 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.367095 11445 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:54.368114 12508 raft_consensus.cc:993] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:54.368168 11335 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:54.368615 12508 raft_consensus.cc:1081] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:54.369175 12598 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:54.370262 12598 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:54.370829 11364 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:54.371338 11364 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.371153 12598 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.371754 11364 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.372126 11516 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:54.379194 11364 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.381436 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:54.381932 11445 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.382561 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:54.383055 11516 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:54.383307 11516 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:54.384065 11516 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.385020 11516 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.387696 11364 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:54.387790 11263 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:54.389063 11445 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:54.390228 11335 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:54.390712 11516 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:54.390993 12598 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:54.391543 12598 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:54.392334 12598 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:54.405192 11263 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:54.523871 12539 raft_consensus.cc:993] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:54.526263 12489 raft_consensus.cc:1081] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:54.531173 12068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54370
I20260709 13:14:54.531709 12068 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:54.531955 12068 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.535880 12068 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:54.537330 12068 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:54.537952 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:54.538214 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:54.538444 11368 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:54.538759 11516 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.538774 11368 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:54.539630 11368 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:54.540422 11368 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:54.543560 11516 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:54.544533 12032 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:54.545179 12553 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:54.545605 12553 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:54.545825 11368 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:54.546342 12553 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:54.564033 11262 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:54.576220 11516 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.577219 12571 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.589380 11368 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.590899 12571 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.748891 11516 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:54.750097 12551 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:54.758883 11364 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:54.759910 12551 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:54.788290 11516 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.789480 12640 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.798475 11445 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.801957 12640 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.898052 11445 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.899288 12638 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:54.913102 11519 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:54.914740 12489 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.011451 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:55.060598 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:55.083345 11443 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:14:55.090247 11409 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad3ab48ae4bf4452a19c99c71fd014f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:55.091959 11368 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:55.090539 11519 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:55.092694 11364 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:55.093367 12571 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.094393 12551 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:55.095628 12656 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.103854 11445 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:14:55.104367 11445 raft_consensus.cc:606] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:55.107455 12571 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.109829 11364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435f2e223e784d188897ac4adbc34211"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:55.110379 11364 raft_consensus.cc:606] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:55.109184 11366 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.115495 11516 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:55.116931 12551 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:55.118729 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:55.119272 11516 raft_consensus.cc:606] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:55.122005 11514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:55.122547 11514 raft_consensus.cc:606] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:55.122390 11511 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.125264 11514 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:55.125792 11514 raft_consensus.cc:606] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:55.130203 11364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:55.130903 11364 raft_consensus.cc:606] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:14:55.131955 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:55.133504 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:14:55.150890 11316 auto_rebalancer.cc:629] Failed to schedule move for tablet 6f3368fb0be64c3b92da009af48a1854: Remote error: Service unavailable: leader transfer in progress
I20260709 13:14:55.156649 11519 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:55.164595 11263 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.184533 11368 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:55.186509 12654 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.184533 11445 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:14:55.191148 11483 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 988262f8a4a846be8b350d9d99c4a730. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:55.192804 12654 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.213971 12559 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.222096 11445 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.229837 11368 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.235828 11366 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:14:55.250031 11334 consensus_peers.cc:597] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8ec7d9e61ab4ff5bd59a37021d68dba. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:55.250919 11445 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:55.252097 11519 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:14:55.252518 12639 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.253545 12638 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:55.256363 11262 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.278435 12639 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.279719 12638 raft_consensus.cc:993] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:55.280292 12638 raft_consensus.cc:1081] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:55.281931 11443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:14:55.282418 11443 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:55.282773 11443 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:55.287932 11519 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.289685 11445 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.288892 11443 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.294006 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:55.294569 11366 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:55.298183 11443 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:55.297168 11366 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:55.299261 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:55.300043 11516 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:55.299652 11366 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.308429 12559 raft_consensus.cc:993] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:55.308871 12559 raft_consensus.cc:1081] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:55.309120 11366 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:55.311578 11368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:55.312146 11368 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:55.312500 11368 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.315809 11366 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:14:55.317221 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:55.318162 12571 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:55.318681 12571 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:55.319540 12571 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.323001 12508 raft_consensus.cc:993] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:55.323496 12508 raft_consensus.cc:1081] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:55.324350 12656 raft_consensus.cc:993] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:14:55.364439 11368 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.304981 11262 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:14:55.380914 11366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:55.384938 11366 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:55.385354 11366 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.386723 11368 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:55.390993 12656 raft_consensus.cc:1081] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:14:55.392460 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:55.393002 11519 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:55.393275 11519 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:55.393972 11519 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.394767 11519 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.400347 11511 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36658
I20260709 13:14:55.400936 11511 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:55.401286 11511 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:55.401960 11519 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:55.402413 11366 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:55.404757 11366 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:14:55.407528 11511 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.409458 11511 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:55.410084 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:14:55.457417 12068 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.419694 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:55.459714 12639 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:55.460222 12639 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:55.461117 12639 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.415315 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:55.485460 11366 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:55.381830 11516 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:14:55.441180 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:55.488794 11440 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.421478 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:55.493527 11519 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:55.493924 11519 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:55.494630 11519 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:55.495849 11519 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.496215 11440 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:55.502435 11519 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:55.507463 12672 raft_consensus.cc:993] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:55.508098 12672 raft_consensus.cc:1081] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:55.510866 12069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:14:55.511718 12069 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:55.512233 12069 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.515087 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:55.515764 11437 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:55.516074 11437 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:55.516773 11437 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.518136 11437 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:55.526953 11437 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:14:55.530071 12639 raft_consensus.cc:993] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:55.535823 12639 raft_consensus.cc:1081] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:14:55.536911 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:55.537997 12489 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:55.538393 12489 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:55.539134 12069 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:55.540061 12067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "435f2e223e784d188897ac4adbc34211"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54370
I20260709 13:14:55.564404 12067 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:55.539126 12489 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:55.564869 12067 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.579779 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:55.580153 12069 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:55.580533 11440 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.581204 12068 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:55.585705 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:55.588074 12672 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:55.590246 12672 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:55.591939 12672 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:55.603116 12067 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:55.673125 12067 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:55.534471 11366 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:14:55.801515 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:55.850512 11368 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:55.854244 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:55.861860 11266 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:55.867074 11440 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:55.927223 11364 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:55.927667 11364 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:55.928745 11364 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:55.930385 11364 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.931608 11437 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:55.954595 11437 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:55.962445 11364 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:14:55.967650 12033 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:55.971689 12551 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:55.972266 12551 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:55.976178 11437 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:55.985757 11437 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:55.986507 12551 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:55.994585 12677 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:14:56.005300 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:56.006091 11516 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:56.006448 11516 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:56.009259 11516 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:14:56.010043 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 424324us
I20260709 13:14:56.011251 11516 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:56.039014 11443 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:56.070012 12685 ts_tablet_manager.cc:933] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:56.092049 12640 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:14:56.115374 11268 catalog_manager.cc:5697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 3, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:56.165186 12685 tablet_copy_client.cc:323] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:14:56.189277 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:56.195933 11516 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
W20260709 13:14:56.201874 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 71974.1us
I20260709 13:14:56.205142 11514 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:56.227187 12696 ts_tablet_manager.cc:933] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:14:56.228976 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 56241.1us
I20260709 13:14:56.232245 11262 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:56.239217 12034 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:14:56.255513 12698 ts_tablet_manager.cc:933] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:14:56.268697 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet ad3ab48ae4bf4452a19c99c71fd014f1 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:14:56.269403 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet ad3ab48ae4bf4452a19c99c71fd014f1 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-ad3ab48ae4bf4452a19c99c71fd014f1
I20260709 13:14:56.268703 12692 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:56.294663 11511 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:56.282087 12692 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:14:56.296644 12692 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:56.308070 11266 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:56.328019 11368 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:56.328408 12640 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:56.329363 11455 tablet_copy_source_session.cc:215] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:56.330021 12068 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:56.329401 11440 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:56.334328 12644 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:14:56.335636 12639 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:56.336760 12685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad3ab48ae4bf4452a19c99c71fd014f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:56.352574 11514 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:56.356249 12694 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:56.367552 12508 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:14:56.369426 12559 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:56.325757 12698 tablet_copy_client.cc:323] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:14:56.349993 12696 tablet_copy_client.cc:323] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:14:56.411808 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 988262f8a4a846be8b350d9d99c4a730 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:14:56.412374 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 988262f8a4a846be8b350d9d99c4a730 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-988262f8a4a846be8b350d9d99c4a730
I20260709 13:14:56.421451 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet b8ec7d9e61ab4ff5bd59a37021d68dba from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
W20260709 13:14:56.432178 12706 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:14:56.438759 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet b8ec7d9e61ab4ff5bd59a37021d68dba from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-b8ec7d9e61ab4ff5bd59a37021d68dba
I20260709 13:14:56.449710 11263 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:56.494324 11529 tablet_copy_source_session.cc:215] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:56.512800 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988262f8a4a846be8b350d9d99c4a730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:56.522549 11380 tablet_copy_source_session.cc:215] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:56.526237 12698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ec7d9e61ab4ff5bd59a37021d68dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:56.525458 11442 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:56.532120 11268 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:56.534731 12692 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:56.563899 11369 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:56.566146 12551 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:56.610693 12685 tablet_copy_client.cc:806] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:56.636315 12685 tablet_copy_client.cc:670] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:56.667038 12685 tablet_copy_client.cc:538] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:14:56.679114 11299 debug-util.cc:398] Leaking SignalData structure 0x7b0800041500 after lost signal to thread 10260
W20260709 13:14:56.680410 11299 debug-util.cc:398] Leaking SignalData structure 0x7b0800034c60 after lost signal to thread 11302
W20260709 13:14:56.681630 11299 debug-util.cc:398] Leaking SignalData structure 0x7b0800344d20 after lost signal to thread 11394
W20260709 13:14:56.682719 11299 debug-util.cc:398] Leaking SignalData structure 0x7b0800335aa0 after lost signal to thread 11469
W20260709 13:14:56.684062 11299 debug-util.cc:398] Leaking SignalData structure 0x7b08003629e0 after lost signal to thread 11543
W20260709 13:14:56.685195 11299 debug-util.cc:398] Leaking SignalData structure 0x7b08002d88e0 after lost signal to thread 12093
I20260709 13:14:56.685912 11436 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:56.689342 12692 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:14:56.710711 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44201.7us
I20260709 13:14:56.732581 12698 tablet_copy_client.cc:806] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:56.733331 12698 tablet_copy_client.cc:670] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:56.739012 12696 tablet_copy_client.cc:806] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:56.743204 11515 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:56.744473 12698 tablet_copy_client.cc:538] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:56.745039 12692 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:56.739785 12696 tablet_copy_client.cc:670] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:56.774915 12696 tablet_copy_client.cc:538] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:56.803926 12685 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:56.816045 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:56.820371 12698 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:56.836408 12685 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:56.839366 12685 tablet_bootstrap.cc:492] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:56.841531 12685 ts_tablet_manager.cc:1403] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.007s
I20260709 13:14:56.822254 12696 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:14:56.863715 12685 raft_consensus.cc:359] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:56.865176 12685 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:56.865931 12685 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:56.868958 12685 ts_tablet_manager.cc:1434] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:14:56.873745 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-ad3ab48ae4bf4452a19c99c71fd014f1 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:14:56.874274 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-ad3ab48ae4bf4452a19c99c71fd014f1 on tablet ad3ab48ae4bf4452a19c99c71fd014f1 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:56.887569 12698 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:56.888557 12698 tablet_bootstrap.cc:492] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:56.889328 12698 ts_tablet_manager.cc:1403] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.069s	user 0.024s	sys 0.012s
I20260709 13:14:56.892568 12698 raft_consensus.cc:359] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:56.898301 12698 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:56.898396 12696 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:56.899657 12696 tablet_bootstrap.cc:492] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:14:56.899178 12698 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:56.900472 12696 ts_tablet_manager.cc:1403] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.078s	user 0.018s	sys 0.011s
I20260709 13:14:56.901794 12698 ts_tablet_manager.cc:1434] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:14:56.905967 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-b8ec7d9e61ab4ff5bd59a37021d68dba received from {username='slave'} at 127.0.0.1:47990
I20260709 13:14:56.906435 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-b8ec7d9e61ab4ff5bd59a37021d68dba on tablet b8ec7d9e61ab4ff5bd59a37021d68dba with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:56.908437 12696 raft_consensus.cc:359] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:56.909345 12696 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:14:56.910038 12696 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:14:56.916834 12696 ts_tablet_manager.cc:1434] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:14:56.920782 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-988262f8a4a846be8b350d9d99c4a730 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:14:56.921325 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-988262f8a4a846be8b350d9d99c4a730 on tablet 988262f8a4a846be8b350d9d99c4a730 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:14:56.922279 12066 raft_consensus.cc:1217] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:57.001718 11515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:57.002410 11515 raft_consensus.cc:606] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:57.010030 11442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:14:57.010598 11442 raft_consensus.cc:606] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:57.013772 11369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:57.014282 11369 raft_consensus.cc:606] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:57.015655 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:57.017475 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:57.118856 12068 raft_consensus.cc:1217] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:57.158584 12068 raft_consensus.cc:1217] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:14:57.291770 12717 raft_consensus.cc:993] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:14:57.292302 12717 raft_consensus.cc:1081] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:14:57.293895 11516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
I20260709 13:14:57.294536 11516 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:57.294888 11516 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:57.301766 11516 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.305332 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:57.306084 11442 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:57.312650 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:57.313309 11370 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:57.313630 11370 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:57.318463 11370 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.319820 11370 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:57.324816 11516 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:57.326840 11442 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:14:57.328625 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:57.329667 12654 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:57.330446 11370 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:14:57.331843 12654 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:57.333746 12654 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.362138 12517 raft_consensus.cc:1064] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
W20260709 13:14:57.364071 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.365586 12517 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:57.398332 12066 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:57.401638 11516 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:57.402885 11370 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:57.404635 12680 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:57.406440 12680 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:57.412691 12679 raft_consensus.cc:993] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:14:57.413857 12679 raft_consensus.cc:1081] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:14:57.414719 12644 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:57.416481 11515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36658
I20260709 13:14:57.416996 11515 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:57.417318 11515 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:57.435153 12683 raft_consensus.cc:993] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:57.435804 12672 raft_consensus.cc:1081] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:57.398981 11262 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 3, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:57.466933 11370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:57.469655 11370 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:57.470027 11370 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:57.468935 12725 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:14:57.480082 11515 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.486927 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:57.487578 11364 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:57.488919 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:57.489517 11442 raft_consensus.cc:3055] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:57.489805 11442 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:57.495589 11442 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.496728 11442 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:57.504556 11364 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:14:57.506134 11442 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:14:57.505985 11515 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:14:57.507077 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:14:57.512343 12672 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:57.512815 12672 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:57.513634 12672 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:14:57.527869 12726 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.079s	user 0.003s	sys 0.000s
I20260709 13:14:57.535490 12680 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:57.530509 11370 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.555245 11370 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:57.555830 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:57.556499 11436 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:57.562208 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:57.562867 11516 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:57.563128 11516 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:57.564849 11516 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.579931 11516 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:57.580019 12508 consensus_queue.cc:1175] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6e87b213a4554d94825aceec352fc094" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:14:57.569820 11436 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:57.587746 11335 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:57.590083 12687 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:57.590564 12687 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:57.591439 12687 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:57.570248 12066 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:57.608561 12717 raft_consensus.cc:1064] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:57.586501 11363 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:57.657253 11515 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:57.685652 12717 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:57.692148 12672 raft_consensus.cc:1064] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:14:57.699586 12672 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:57.701457 11516 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:57.719158 11268 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:57.757289 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 77616.6us
I20260709 13:14:57.780489 11369 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:57.782472 12683 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:57.802039 11263 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:57.828173 11442 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:57.834477 11262 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:57.846529 12066 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:57.851509 11363 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:57.857300 12644 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:57.874826 12727 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:57.897303 12644 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:57.906095 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ad3ab48ae4bf4452a19c99c71fd014f1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:57.917935 11263 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:57.940399 11516 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:57.942057 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet ad3ab48ae4bf4452a19c99c71fd014f1 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:57.942189 12717 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:57.958776 12741 tablet_replica.cc:333] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:57.960130 12741 raft_consensus.cc:2243] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.963583 12066 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:57.969761 11363 raft_consensus.cc:2955] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:57.988065 12068 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:57.989494 11442 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:57.991053 11367 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:58.003757 12068 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:58.004653 11442 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:58.011157 12687 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:58.015257 12654 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:58.017494 11364 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.018568 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:58.020208 11442 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:14:58.025090 12654 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:58.028854 12508 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:58.003639 12741 raft_consensus.cc:2272] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.034480 12741 ts_tablet_manager.cc:1916] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:58.034325 12687 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:58.034588 11517 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.037423 12683 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:58.040041 12654 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:58.065375 12687 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:14:58.110271 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46894.4us
W20260709 13:14:58.132596 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 72424.7us
I20260709 13:14:58.168259 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:58.319839 12752 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:14:58.219492 11436 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:14:58.416960 12748 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 13:14:58.424161 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 207433us
I20260709 13:14:58.424481 12741 ts_tablet_manager.cc:1929] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:58.124586 12687 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:14:58.521930 12766 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:14:58.544117 11515 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.572171 12741 log.cc:1199] T ad3ab48ae4bf4452a19c99c71fd014f1 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/ad3ab48ae4bf4452a19c99c71fd014f1
W20260709 13:14:58.633930 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 161484us
I20260709 13:14:58.637866 12672 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:14:58.678589 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68924.8us
I20260709 13:14:58.683426 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet ad3ab48ae4bf4452a19c99c71fd014f1 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:58.688690 11368 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.706121 12066 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.731513 11442 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:58.741532 11436 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.746532 11263 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.755427 11262 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.770553 11511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:58.771158 11511 raft_consensus.cc:606] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:58.780625 11515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:14:58.781217 11515 raft_consensus.cc:606] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:14:58.767571 12068 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.785634 12687 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:58.786518 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:58.787122 11370 raft_consensus.cc:606] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:14:58.787567 12773 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:14:58.789330 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:58.790916 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:14:58.797886 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 71475.6us
I20260709 13:14:58.808441 12773 raft_consensus.cc:993] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:58.809631 12758 raft_consensus.cc:1081] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:58.827802 11436 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.840044 11441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51474
I20260709 13:14:58.840679 11441 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.841051 11441 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.851876 11441 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:58.854390 11441 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:14:58.856401 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:14:58.857539 12068 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.857678 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:58.858505 11511 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.858791 11511 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:58.859704 11511 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:58.860844 11511 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.870550 11511 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:58.871781 11408 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:58.875730 12769 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.876287 12769 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:58.877235 12769 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:58.895534 11370 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:58.902082 12066 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:58.905308 12068 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:58.908527 11443 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:58.910877 12508 raft_consensus.cc:993] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:58.911481 12508 raft_consensus.cc:1081] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:14:58.910528 12778 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:14:58.914608 11368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:14:58.915103 11368 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.915427 11368 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.914602 12778 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:14:58.927104 11368 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:58.934219 11368 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:14:58.934487 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:14:58.935473 11443 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.936149 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:14:58.936672 11515 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:58.936909 11515 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:14:58.939267 11515 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:58.940482 11515 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.936030 12762 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.938163 11436 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:58.947431 11443 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:58.957677 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:58.948670 11515 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:14:58.951122 11511 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:58.960125 12770 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:58.960592 12770 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:58.967391 11368 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:58.969200 12782 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:58.961452 12770 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:14:58.973497 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8ec7d9e61ab4ff5bd59a37021d68dba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:58.977135 12638 raft_consensus.cc:993] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:58.976255 11262 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:58.977599 12638 raft_consensus.cc:1081] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:14:58.978974 12068 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:14:58.982018 11443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:14:58.982396 11443 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:58.982631 11443 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:58.985630 12783 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:58.992385 12068 raft_consensus.cc:2955] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:59.003986 11443 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:59.003799 12743 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:59.025120 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:14:59.026283 11369 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:59.026597 11369 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:14:59.027293 11369 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:59.030897 11369 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:59.030757 12068 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:59.033505 11443 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:14:59.034744 11370 raft_consensus.cc:2955] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:59.046098 11266 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:59.066847 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:14:59.073475 12066 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:59.076894 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet b8ec7d9e61ab4ff5bd59a37021d68dba with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:14:59.077845 12741 tablet_replica.cc:333] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:14:59.078835 12741 raft_consensus.cc:2243] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:59.079510 12741 raft_consensus.cc:2272] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:59.083073 12741 ts_tablet_manager.cc:1916] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:59.069031 11369 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:59.092443 12066 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:14:59.092710 11263 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:14:59.097873 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:14:59.102607 12767 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:14:59.103127 12767 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:14:59.106961 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 988262f8a4a846be8b350d9d99c4a730 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:14:59.107404 11268 catalog_manager.cc:5697] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:59.109128 12767 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:14:59.115756 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 988262f8a4a846be8b350d9d99c4a730 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:59.136734 12741 ts_tablet_manager.cc:1929] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:59.138952 12741 log.cc:1199] T b8ec7d9e61ab4ff5bd59a37021d68dba P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/b8ec7d9e61ab4ff5bd59a37021d68dba
I20260709 13:14:59.143755 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet b8ec7d9e61ab4ff5bd59a37021d68dba (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:59.150321 12794 tablet_replica.cc:333] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:14:59.151580 12794 raft_consensus.cc:2243] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:59.152396 12794 raft_consensus.cc:2272] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:59.155009 11266 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:59.156383 12794 ts_tablet_manager.cc:1916] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:59.186514 12794 ts_tablet_manager.cc:1929] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:14:59.195588 12794 log.cc:1199] T 988262f8a4a846be8b350d9d99c4a730 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/988262f8a4a846be8b350d9d99c4a730
I20260709 13:14:59.197362 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 988262f8a4a846be8b350d9d99c4a730 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:14:59.460528 11515 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:59.461774 12776 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:59.484371 12065 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:59.485551 11437 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.486600 12769 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:59.491606 12770 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:59.497073 12066 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:59.501281 12769 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:59.526960 11368 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:14:59.530510 11511 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:59.531766 12778 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:59.548530 12776 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:59.792001 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:59.868597 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:59.922663 11436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:14:59.923379 11436 raft_consensus.cc:606] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:14:59.927783 11436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:14:59.928392 11436 raft_consensus.cc:606] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:14:59.932515 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:14:59.933054 11370 raft_consensus.cc:606] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:14:59.934149 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:59.935804 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:00.094908 12776 raft_consensus.cc:993] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:00.095348 12767 raft_consensus.cc:1081] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:00.096707 12066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:35176
I20260709 13:15:00.097146 12066 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:00.097489 12066 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.102232 12066 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:00.103902 12066 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:00.104635 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:00.105043 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:00.105278 11436 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:00.105599 11515 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.105695 11436 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:00.106667 11436 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:00.109254 11436 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.114787 11515 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:00.116418 11436 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:00.116554 12032 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:00.117419 12780 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:00.117877 12780 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:00.118726 12780 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:00.124101 12776 raft_consensus.cc:993] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:00.124666 12776 raft_consensus.cc:1081] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:00.128083 11370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:15:00.128674 11370 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:00.129004 11370 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.144950 11370 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:00.150215 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:00.150780 11436 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:00.151055 11436 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:00.151778 11436 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:00.152747 11436 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.157783 11263 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:00.163875 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:00.164446 11515 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.169538 11436 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:00.170326 11370 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:00.177779 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:00.180246 12770 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:00.180719 12770 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:00.181615 12770 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:00.184888 12778 raft_consensus.cc:993] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:15:00.185312 12778 raft_consensus.cc:1081] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:15:00.192279 11511 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
I20260709 13:15:00.192809 11511 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:00.193167 11511 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.197136 11437 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:00.205027 11511 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:00.206930 11511 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:00.208271 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:00.213773 11441 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.209444 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:00.219676 11369 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:00.219987 11369 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:00.220696 11369 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:00.222113 11369 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:00.174476 11515 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:00.219094 11441 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:00.238749 11484 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:00.241631 12672 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:00.242146 12672 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:00.243024 12672 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:00.247438 11370 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:00.249997 12776 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.254032 11369 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:00.243220 11263 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:00.257875 11409 consensus_peers.cc:597] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ea18eac305644ed8d44618c2c0a8d0d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.275709 12776 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.284934 11515 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:00.288725 12769 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.293557 11370 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.339969 11511 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:00.360426 11268 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:00.363823 11515 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.365473 11437 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:00.367440 12817 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:00.377767 11266 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.385838 11370 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:00.389204 12818 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.389897 11441 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.400780 12672 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:15:00.406286 11483 consensus_peers.cc:597] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 572d79fba792428db7fcc1994fff81f7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.421330 11370 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.426343 11268 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.430764 11441 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:15:00.454241 11409 consensus_peers.cc:597] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c1dd88ce9a04ff59ee55a115686f148. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.455217 11515 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:00.456626 12767 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.457832 11369 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:00.466200 12767 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.469956 12769 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.545157 11515 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.545401 11263 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:00.548105 11369 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:00.567955 11367 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.576824 11511 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:00.578065 11334 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbbdba17cce24fe7923cf755be2d3e0a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:00.581516 12755 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.587173 11441 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:00.590322 12755 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.595012 12778 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.607569 11441 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:00.610190 11511 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.615525 12770 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.648784 11515 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:00.650962 12831 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.676389 11266 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.686200 12835 ts_tablet_manager.cc:933] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:00.689417 11436 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:00.707345 11517 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:00.709920 12760 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:00.744117 11441 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:00.745946 12835 tablet_copy_client.cc:323] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:15:00.749475 12780 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:00.754458 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 1ea18eac305644ed8d44618c2c0a8d0d from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:15:00.754969 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 1ea18eac305644ed8d44618c2c0a8d0d from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-1ea18eac305644ed8d44618c2c0a8d0d
I20260709 13:15:00.762404 11455 tablet_copy_source_session.cc:215] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:00.782224 12835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ea18eac305644ed8d44618c2c0a8d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:00.802050 11443 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:00.805886 12818 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:00.817683 11368 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:00.820531 12818 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:00.924712 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 81507.8us
I20260709 13:15:00.938416 12847 ts_tablet_manager.cc:933] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:00.939193 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:00.953047 12848 ts_tablet_manager.cc:933] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:00.971812 12835 tablet_copy_client.cc:806] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:00.972379 12847 tablet_copy_client.cc:323] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:15:00.972478 12835 tablet_copy_client.cc:670] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:00.974643 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 9c1dd88ce9a04ff59ee55a115686f148 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:15:00.975131 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 9c1dd88ce9a04ff59ee55a115686f148 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-9c1dd88ce9a04ff59ee55a115686f148
I20260709 13:15:00.983839 11455 tablet_copy_source_session.cc:215] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:00.989624 12847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1dd88ce9a04ff59ee55a115686f148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:00.985100 12835 tablet_copy_client.cc:538] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:01.006229 12848 tablet_copy_client.cc:323] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:15:01.007862 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 572d79fba792428db7fcc1994fff81f7 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:15:01.008421 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 572d79fba792428db7fcc1994fff81f7 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-572d79fba792428db7fcc1994fff81f7
I20260709 13:15:01.015789 11529 tablet_copy_source_session.cc:215] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:01.018105 12835 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:01.023135 12848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 572d79fba792428db7fcc1994fff81f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.042546 12848 tablet_copy_client.cc:806] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:01.043114 12848 tablet_copy_client.cc:670] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:01.055650 12848 tablet_copy_client.cc:538] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:01.064682 12848 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:01.075768 12847 tablet_copy_client.cc:806] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:01.076439 12847 tablet_copy_client.cc:670] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:01.080129 12847 tablet_copy_client.cc:538] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:01.086575 12835 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:01.087392 12835 tablet_bootstrap.cc:492] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:01.088219 12835 ts_tablet_manager.cc:1403] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.070s	user 0.025s	sys 0.006s
I20260709 13:15:01.091977 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:01.094061 12847 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:01.104149 12835 raft_consensus.cc:359] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.116482 12835 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:01.124473 12835 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.138367 12835 ts_tablet_manager.cc:1434] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.001s
I20260709 13:15:01.153911 12848 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:01.155987 12848 tablet_bootstrap.cc:492] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:01.156656 12848 ts_tablet_manager.cc:1403] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.092s	user 0.022s	sys 0.005s
I20260709 13:15:01.159799 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-1ea18eac305644ed8d44618c2c0a8d0d received from {username='slave'} at 127.0.0.1:41514
I20260709 13:15:01.160270 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-1ea18eac305644ed8d44618c2c0a8d0d on tablet 1ea18eac305644ed8d44618c2c0a8d0d with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:01.160867 11517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:15:01.162612 11517 raft_consensus.cc:606] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:01.160748 12848 raft_consensus.cc:359] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.166066 12848 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:01.166791 12848 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.171633 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:01.172132 12835 ts_tablet_manager.cc:933] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:01.172282 11368 raft_consensus.cc:606] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:15:01.181938 12848 ts_tablet_manager.cc:1434] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:15:01.183048 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:01.184839 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:01.199498 12835 tablet_copy_client.cc:323] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:15:01.201303 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet dbbdba17cce24fe7923cf755be2d3e0a from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:15:01.201865 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet dbbdba17cce24fe7923cf755be2d3e0a from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-dbbdba17cce24fe7923cf755be2d3e0a
I20260709 13:15:01.202991 12065 raft_consensus.cc:1217] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:01.184762 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-572d79fba792428db7fcc1994fff81f7 received from {username='slave'} at 127.0.0.1:48448
I20260709 13:15:01.216936 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-572d79fba792428db7fcc1994fff81f7 on tablet 572d79fba792428db7fcc1994fff81f7 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:01.219062 11380 tablet_copy_source_session.cc:215] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:01.222892 12835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbbdba17cce24fe7923cf755be2d3e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:01.227665 12847 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:01.228564 12847 tablet_bootstrap.cc:492] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:01.229206 12847 ts_tablet_manager.cc:1403] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.135s	user 0.025s	sys 0.004s
I20260709 13:15:01.238853 12835 tablet_copy_client.cc:806] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:01.239553 12835 tablet_copy_client.cc:670] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:01.240628 12847 raft_consensus.cc:359] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.241468 12847 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:01.242307 12847 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.243996 12835 tablet_copy_client.cc:538] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:01.245148 12847 ts_tablet_manager.cc:1434] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:15:01.248971 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-9c1dd88ce9a04ff59ee55a115686f148 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:15:01.249372 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-9c1dd88ce9a04ff59ee55a115686f148 on tablet 9c1dd88ce9a04ff59ee55a115686f148 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:01.253584 12830 raft_consensus.cc:993] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:15:01.254016 12830 raft_consensus.cc:1081] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:15:01.259822 11441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:15:01.260339 11441 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:01.260667 11441 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:01.268123 12835 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:01.277007 11441 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:01.279505 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:01.280055 11368 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:01.280351 11368 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:01.281083 11368 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:01.282145 11368 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:01.282706 12065 raft_consensus.cc:1217] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:15:01.313104 11368 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:01.314252 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:01.315119 12826 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:01.315176 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:01.315670 12826 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:01.315771 11517 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:01.316514 12826 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:01.324404 11517 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:01.325995 11441 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:01.335330 12835 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:01.336371 12835 tablet_bootstrap.cc:492] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:01.337392 12835 ts_tablet_manager.cc:1403] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.070s	user 0.043s	sys 0.024s
I20260709 13:15:01.340785 12835 raft_consensus.cc:359] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.341931 12835 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:01.342519 12835 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:01.344812 12835 ts_tablet_manager.cc:1434] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:15:01.347190 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-dbbdba17cce24fe7923cf755be2d3e0a received from {username='slave'} at 127.0.0.1:47990
I20260709 13:15:01.347646 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-dbbdba17cce24fe7923cf755be2d3e0a on tablet dbbdba17cce24fe7923cf755be2d3e0a with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:01.358228 11268 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:01.377765 12743 raft_consensus.cc:993] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:01.378283 12743 raft_consensus.cc:1081] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:01.380208 11369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:15:01.380775 11369 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:01.381196 11369 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:01.388021 11369 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:01.390023 11369 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:01.391265 12065 raft_consensus.cc:1217] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:01.396925 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:01.397521 11517 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:01.397814 11517 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:01.398429 11517 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:01.401554 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:01.402159 12065 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:01.404259 11517 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:01.412526 11517 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:01.412730 12065 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:01.413777 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:01.414535 12755 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:01.415037 12755 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:01.415930 12755 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:01.429281 11266 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:01.586395 12826 raft_consensus.cc:1064] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:01.588950 12826 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:01.603113 12068 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:01.605929 12769 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:01.613646 11511 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:01.614729 11368 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:01.621701 12776 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:01.625188 12065 raft_consensus.cc:1217] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:01.633956 12769 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:01.658550 12826 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.662384 11511 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.675499 12068 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.682046 11367 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.689669 12743 raft_consensus.cc:1064] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:01.696619 12743 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:01.729259 11266 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.770548 11436 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:01.778285 12068 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:01.780365 12865 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:01.783389 11511 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:01.783581 12068 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:01.785980 11367 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:01.788757 11436 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:01.785854 12672 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:01.792495 12818 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:01.794520 12866 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.798702 12068 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:15:01.835598 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46388.7us
I20260709 13:15:01.836867 12865 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:01.843834 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1ea18eac305644ed8d44618c2c0a8d0d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:01.845124 11511 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:01.851291 11268 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:01.852584 11511 raft_consensus.cc:2955] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:15:01.859968 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 49979.3us
I20260709 13:15:01.863284 11516 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:01.871026 12778 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:01.875669 12868 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:01.880542 12672 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.885720 11367 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.891515 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ea18eac305644ed8d44618c2c0a8d0d with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37074
I20260709 13:15:01.903272 12068 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.908167 12873 tablet_replica.cc:333] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:15:01.909837 12873 raft_consensus.cc:2243] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.909943 11369 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:01.913853 12769 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:01.974023 12873 raft_consensus.cc:2272] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.931964 11266 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.937577 12776 raft_consensus.cc:1064] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:01.977931 12873 ts_tablet_manager.cc:1916] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:01.979215 12776 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:01.964753 11441 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:01.984513 12769 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:01.970649 12068 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:01.989404 12778 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:02.015499 12873 ts_tablet_manager.cc:1929] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:02.020794 12873 log.cc:1199] T 1ea18eac305644ed8d44618c2c0a8d0d P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/1ea18eac305644ed8d44618c2c0a8d0d
I20260709 13:15:02.025544 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 1ea18eac305644ed8d44618c2c0a8d0d (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:02.027557 12065 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.028612 11516 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.030704 11369 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.032176 12776 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:02.034032 11516 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:02.033955 12866 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:02.036329 12769 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:02.049504 12066 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:02.051242 12672 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:02.060850 12818 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.077105 11443 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:02.079139 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 572d79fba792428db7fcc1994fff81f7 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:02.073369 12066 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.090683 12818 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:02.096846 12755 raft_consensus.cc:1064] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:02.121102 11441 raft_consensus.cc:2955] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.140892 12826 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.145125 12069 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.185813 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:02.152504 11369 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.209949 11262 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 4 to 5, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.222311 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 572d79fba792428db7fcc1994fff81f7 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37074
I20260709 13:15:02.223269 12873 tablet_replica.cc:333] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:15:02.224360 12873 raft_consensus.cc:2243] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:02.225078 12873 raft_consensus.cc:2272] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.228626 12873 ts_tablet_manager.cc:1916] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:15:02.229514 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 110410us
I20260709 13:15:02.237946 12755 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:02.251017 11511 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.271750 11266 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.293187 11443 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:02.300473 12066 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:02.301863 12826 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:02.303087 11369 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:02.304518 12826 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:02.308418 12769 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.310001 11517 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.311228 11441 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.311548 12885 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:02.312736 12778 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:02.320854 12069 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.327359 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9c1dd88ce9a04ff59ee55a115686f148 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:02.328955 11368 raft_consensus.cc:2955] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.328943 12066 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:02.343767 12873 ts_tablet_manager.cc:1929] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:15:02.344264 12873 log.cc:1199] T 572d79fba792428db7fcc1994fff81f7 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/572d79fba792428db7fcc1994fff81f7
I20260709 13:15:02.344715 12885 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:02.345767 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 572d79fba792428db7fcc1994fff81f7 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:02.349186 11266 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.380080 12755 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.386663 11436 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.386955 12069 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.400573 11516 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.442279 11262 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.444424 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c1dd88ce9a04ff59ee55a115686f148 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:15:02.471432 12892 tablet_replica.cc:333] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:15:02.472501 12892 raft_consensus.cc:2243] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:02.473235 12892 raft_consensus.cc:2272] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.480037 11369 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:02.483671 12892 ts_tablet_manager.cc:1916] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:02.500620 12069 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:02.500852 11511 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:02.502130 12830 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:02.508354 12755 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:02.517318 12830 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.523985 11511 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.530110 12069 raft_consensus.cc:2955] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.538523 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:02.550173 12892 ts_tablet_manager.cc:1929] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:02.586977 12892 log.cc:1199] T 9c1dd88ce9a04ff59ee55a115686f148 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/9c1dd88ce9a04ff59ee55a115686f148
I20260709 13:15:02.562357 11263 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 3 to 4, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:02.551357 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dbbdba17cce24fe7923cf755be2d3e0a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:02.606150 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 9c1dd88ce9a04ff59ee55a115686f148 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:02.616773 11436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:15:02.617362 11436 raft_consensus.cc:606] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:02.626745 11369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:02.627362 11369 raft_consensus.cc:606] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:02.631374 11369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:02.632092 11369 raft_consensus.cc:606] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:15:02.633316 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:15:02.634945 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:02.644183 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet dbbdba17cce24fe7923cf755be2d3e0a with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51468
I20260709 13:15:02.652478 12893 tablet_replica.cc:333] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:15:02.653662 12893 raft_consensus.cc:2243] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:02.654434 12893 raft_consensus.cc:2272] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.658736 12893 ts_tablet_manager.cc:1916] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:02.661145 12826 raft_consensus.cc:993] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:02.661666 12826 raft_consensus.cc:1081] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:02.664587 11369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:15:02.665156 11369 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.665441 11369 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:02.670521 11369 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:02.672847 11369 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:02.673239 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:02.673792 11443 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:02.674096 11443 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:02.674511 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:02.675040 11516 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:02.674784 11443 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:02.675951 11443 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:02.685647 11443 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:02.686794 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:02.686996 12893 ts_tablet_manager.cc:1929] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:02.687538 12893 log.cc:1199] T dbbdba17cce24fe7923cf755be2d3e0a P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/dbbdba17cce24fe7923cf755be2d3e0a
I20260709 13:15:02.689165 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet dbbdba17cce24fe7923cf755be2d3e0a (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:02.687661 12885 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:02.690214 12885 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:02.691063 12885 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:02.693389 11516 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:02.703212 11263 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 3, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:02.761858 12755 raft_consensus.cc:993] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:15:02.762377 12885 raft_consensus.cc:1081] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:15:02.764904 11516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
I20260709 13:15:02.765535 11516 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.766400 11516 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:02.776677 11516 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:02.778090 11516 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:02.778949 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:02.779073 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:02.779613 11369 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:02.779690 11443 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:02.779997 11369 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:02.780731 11369 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:02.781908 11369 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:02.785768 11443 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:02.786885 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:02.787477 12818 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:02.787847 12818 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:02.788631 11369 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:02.788508 12818 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:02.799611 11263 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 3, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.913800 12885 raft_consensus.cc:993] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:02.914350 12755 raft_consensus.cc:1081] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:02.916314 12069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54370
I20260709 13:15:02.917392 12069 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:02.917886 12069 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.924188 12069 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:02.926358 12069 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:02.926463 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:02.927100 11369 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:02.927271 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:02.928355 11443 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.932592 11369 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:02.933089 11443 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:02.933380 11369 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:02.934055 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:02.934605 11369 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:02.935617 12844 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:02.936102 12844 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:02.937667 12844 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:02.941658 11369 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:02.953032 11263 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:03.163852 11443 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:03.164988 12755 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.169728 11369 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:03.171051 12868 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.185873 11511 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:03.188087 12755 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.218839 11436 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:03.226976 12818 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:03.333706 11443 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:03.335068 12844 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:03.379223 11369 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:03.383882 12912 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:03.635812 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:03.729166 11511 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:15:03.737099 11483 consensus_peers.cc:597] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a992bbbf8dd4ccbb70ff8d65e31edae. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:03.738808 11369 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.740952 12868 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.750969 11436 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:03.752406 12906 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.753985 12905 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.761098 11369 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.772203 11263 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:03.779527 11436 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:03.791937 11443 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
W20260709 13:15:03.825536 11409 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5b7e70b18df44b6b0e248fbc3664d96. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:03.832063 11516 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:03.832734 11369 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:03.833639 12922 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:03.835522 12826 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:03.845876 12826 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.872243 11516 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.881538 11369 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:03.888242 11268 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 4, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:03.901331 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46008.1us
I20260709 13:15:03.906920 11367 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.916909 11511 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:03.923017 12885 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:03.924149 11334 consensus_peers.cc:597] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a142ac56caa42a08e0a838dc87ac3b3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:03.924503 11443 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:03.929342 12885 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:03.944522 12755 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.946986 11511 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.948910 11443 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:03.959632 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:03.976903 11263 catalog_manager.cc:5697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:04.037256 11511 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:15:04.043761 11483 consensus_peers.cc:597] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b01b3301ee794d99bcb4feb672ac699f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:04.045450 11367 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:04.046132 11443 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:04.047600 12906 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:04.048537 12818 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:04.068270 11511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:15:04.069028 11511 raft_consensus.cc:606] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:04.073151 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:04.072649 12905 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:04.073840 11370 raft_consensus.cc:606] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:15:04.075913 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:04.077569 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:04.076699 11370 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:04.087067 11443 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:04.098048 11266 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.137945 12755 raft_consensus.cc:993] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:15:04.138386 12755 raft_consensus.cc:1081] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:15:04.142920 11443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:15:04.143584 11443 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:04.144022 11443 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:04.150242 11443 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:04.152588 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:04.152588 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:04.153255 11511 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:04.153232 11370 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:04.153733 11370 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:04.154443 11370 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:04.155478 11370 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:04.160260 11511 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:04.161958 11408 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:04.162603 12925 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:04.163062 12925 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:04.163852 12925 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:04.173386 11370 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:04.178339 11443 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:04.188915 11263 catalog_manager.cc:5697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.247785 12934 ts_tablet_manager.cc:933] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:04.267354 12936 ts_tablet_manager.cc:933] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:04.269971 12934 tablet_copy_client.cc:323] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:15:04.271668 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:15:04.272172 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet 1a992bbbf8dd4ccbb70ff8d65e31edae from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-1a992bbbf8dd4ccbb70ff8d65e31edae
I20260709 13:15:04.283595 11529 tablet_copy_source_session.cc:215] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:04.285332 12936 tablet_copy_client.cc:323] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:15:04.292666 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet e5b7e70b18df44b6b0e248fbc3664d96 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:15:04.293187 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet e5b7e70b18df44b6b0e248fbc3664d96 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-e5b7e70b18df44b6b0e248fbc3664d96
I20260709 13:15:04.309890 11455 tablet_copy_source_session.cc:215] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:04.311669 12934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:04.318490 12936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b7e70b18df44b6b0e248fbc3664d96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:04.325487 12905 raft_consensus.cc:993] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:04.325978 12905 raft_consensus.cc:1081] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:04.331964 11370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:15:04.332463 11370 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:04.332778 11370 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:04.349615 11370 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.356727 11370 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:04.356727 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:04.358851 11443 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:04.358062 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:04.360110 11516 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:04.360430 11516 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:04.361204 11516 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.380745 11516 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:04.400631 12936 tablet_copy_client.cc:806] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:04.401261 12936 tablet_copy_client.cc:670] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:04.403153 12934 tablet_copy_client.cc:806] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:04.405684 12934 tablet_copy_client.cc:670] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:04.406035 11443 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:04.408972 11516 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:04.410493 11335 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:04.415067 12936 tablet_copy_client.cc:538] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:04.415616 12755 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:04.416054 12755 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:04.416787 12755 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.421108 12934 tablet_copy_client.cc:538] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:04.423823 12936 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:04.434106 12934 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:04.445047 11263 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 3, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.472306 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37074
I20260709 13:15:04.472846 11370 raft_consensus.cc:606] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:15:04.477499 12936 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.477988 12938 ts_tablet_manager.cc:933] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:04.478636 12936 tablet_bootstrap.cc:492] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:04.479530 12936 ts_tablet_manager.cc:1403] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.021s
I20260709 13:15:04.483430 12936 raft_consensus.cc:359] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.484289 12936 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:04.484869 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37074
I20260709 13:15:04.485488 11370 raft_consensus.cc:606] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:15:04.484972 12936 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:15:04.486758 11253 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 1a992bbbf8dd4ccbb70ff8d65e31edae already in progress [suppressed 1 similar messages]
I20260709 13:15:04.502609 12934 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.504870 12938 tablet_copy_client.cc:323] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:15:04.505226 12934 tablet_bootstrap.cc:492] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:04.506865 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 8a142ac56caa42a08e0a838dc87ac3b3 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:15:04.507426 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 8a142ac56caa42a08e0a838dc87ac3b3 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-8a142ac56caa42a08e0a838dc87ac3b3
I20260709 13:15:04.508191 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37074
I20260709 13:15:04.508695 11370 raft_consensus.cc:606] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:15:04.509146 12936 ts_tablet_manager.cc:1434] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:15:04.510665 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-e5b7e70b18df44b6b0e248fbc3664d96 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:15:04.511031 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-e5b7e70b18df44b6b0e248fbc3664d96 on tablet e5b7e70b18df44b6b0e248fbc3664d96 with peer b1906ab37ba541f493936685a808c8a9
W20260709 13:15:04.521019 11316 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
I20260709 13:15:04.521384 11380 tablet_copy_source_session.cc:215] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:04.506525 12934 ts_tablet_manager.cc:1403] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.073s	user 0.022s	sys 0.007s
W20260709 13:15:04.521488 11316 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 1a992bbbf8dd4ccbb70ff8d65e31edae already in progress
W20260709 13:15:04.523815 11316 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 1a992bbbf8dd4ccbb70ff8d65e31edae already in progress
I20260709 13:15:04.526489 12934 raft_consensus.cc:359] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.527374 12934 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:04.528468 12934 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.531875 12934 ts_tablet_manager.cc:1434] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.006s
I20260709 13:15:04.537611 12938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a142ac56caa42a08e0a838dc87ac3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:04.539369 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-1a992bbbf8dd4ccbb70ff8d65e31edae received from {username='slave'} at 127.0.0.1:48448
I20260709 13:15:04.539840 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-1a992bbbf8dd4ccbb70ff8d65e31edae on tablet 1a992bbbf8dd4ccbb70ff8d65e31edae with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:04.554742 12938 tablet_copy_client.cc:806] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:04.555346 12938 tablet_copy_client.cc:670] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:04.560734 12938 tablet_copy_client.cc:538] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:04.569818 12938 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:04.591673 12934 ts_tablet_manager.cc:933] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:04.592270 12938 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.593236 12938 tablet_bootstrap.cc:492] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:04.593935 12938 ts_tablet_manager.cc:1403] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.010s
I20260709 13:15:04.596617 11370 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:04.596983 12938 raft_consensus.cc:359] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.598320 12938 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:04.599031 12926 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:04.599291 12938 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.602241 12938 ts_tablet_manager.cc:1434] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:15:04.604200 12934 tablet_copy_client.cc:323] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.67:46667
I20260709 13:15:04.606393 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-8a142ac56caa42a08e0a838dc87ac3b3 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:15:04.606875 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-8a142ac56caa42a08e0a838dc87ac3b3 on tablet 8a142ac56caa42a08e0a838dc87ac3b3 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:04.611971 11529 tablet_copy_service.cc:140] P 34a42aa6aa0a4778a532c6433d70b981: Received BeginTabletCopySession request for tablet b01b3301ee794d99bcb4feb672ac699f from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:48448)
I20260709 13:15:04.612521 11529 tablet_copy_service.cc:161] P 34a42aa6aa0a4778a532c6433d70b981: Beginning new tablet copy session on tablet b01b3301ee794d99bcb4feb672ac699f from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:48448: session id = b1906ab37ba541f493936685a808c8a9-b01b3301ee794d99bcb4feb672ac699f
I20260709 13:15:04.629557 11529 tablet_copy_source_session.cc:215] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:04.641121 12934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b01b3301ee794d99bcb4feb672ac699f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:04.667611 11516 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:04.677377 12934 tablet_copy_client.cc:806] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:04.690266 12934 tablet_copy_client.cc:670] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:04.681916 12926 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:15:04.704627 12934 tablet_copy_client.cc:538] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:04.714085 12934 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:04.768824 12934 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:04.769939 12934 tablet_bootstrap.cc:492] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:04.770597 12934 ts_tablet_manager.cc:1403] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.057s	user 0.018s	sys 0.009s
I20260709 13:15:04.775677 12934 raft_consensus.cc:359] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.776535 12934 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:04.777207 12934 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.780762 12934 ts_tablet_manager.cc:1434] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:15:04.782531 11529 tablet_copy_service.cc:342] P 34a42aa6aa0a4778a532c6433d70b981: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-b01b3301ee794d99bcb4feb672ac699f received from {username='slave'} at 127.0.0.1:48448
I20260709 13:15:04.782967 11529 tablet_copy_service.cc:434] P 34a42aa6aa0a4778a532c6433d70b981: ending tablet copy session b1906ab37ba541f493936685a808c8a9-b01b3301ee794d99bcb4feb672ac699f on tablet b01b3301ee794d99bcb4feb672ac699f with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:04.860085 12066 raft_consensus.cc:1217] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:15:04.893345 11511 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:04.895027 12830 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:04.910709 12885 raft_consensus.cc:993] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:15:04.911224 12885 raft_consensus.cc:1081] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:15:04.914413 11516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
I20260709 13:15:04.914943 12066 raft_consensus.cc:1217] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:04.914956 11516 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:04.918669 11516 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:04.949867 11516 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.954262 11511 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:04.954667 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:04.955220 11443 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:04.961393 12830 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:04.962023 11443 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:04.963351 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:04.964167 12906 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:04.965075 12906 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:04.968132 12906 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:04.980983 11268 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 3 to 4, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 4 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:04.993637 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:04.991513 11516 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:05.002818 11370 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:05.003254 11370 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:04.995225 11443 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:05.004029 11370 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:05.005260 11370 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:05.012596 11370 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:05.024775 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 108199us
I20260709 13:15:05.035199 12066 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Advancing to term 3
I20260709 13:15:05.066128 12066 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:05.078569 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:05.125059 12069 raft_consensus.cc:1217] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:15:05.300266 12959 raft_consensus.cc:1064] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:05.370894 12959 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:05.386721 11436 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:05.395021 12955 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:05.441015 11370 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:05.456956 12818 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:05.473685 11517 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:05.476374 12926 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:05.486183 12066 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Advancing to term 4
I20260709 13:15:05.490985 12068 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:05.492882 12926 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:05.493211 11370 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:05.525597 12926 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:05.560238 12954 raft_consensus.cc:1064] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:05.576994 12066 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:05.578195 12925 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.581118 11367 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.598878 11516 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.629103 12065 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.639765 11266 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 4 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.572589 12954 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:05.655236 12955 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:05.671154 11441 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:05.671810 12065 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:05.673650 12953 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:05.671157 11516 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:05.681056 12885 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:05.682130 11441 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:05.683017 12885 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:05.693861 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:05.706933 12068 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:05.707722 11511 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:05.710201 12964 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:05.714018 12959 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:05.722208 12906 raft_consensus.cc:1064] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:05.724864 12906 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:05.726657 12925 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.750870 11511 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.756035 12969 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.761392 12065 raft_consensus.cc:2955] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.765870 11517 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.770011 11443 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:05.770372 11368 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:05.770766 12068 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:05.771580 12906 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:05.772673 12955 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:05.772980 12069 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.782941 12955 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:05.802398 11268 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.828909 11443 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.833190 11262 catalog_manager.cc:5697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:05.835278 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e5b7e70b18df44b6b0e248fbc3664d96 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:05.841127 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet e5b7e70b18df44b6b0e248fbc3664d96 with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37074
I20260709 13:15:05.853968 12818 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.866983 12987 tablet_replica.cc:333] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:15:05.875829 11437 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.877560 11370 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.896660 12987 raft_consensus.cc:2243] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:05.902148 12987 raft_consensus.cc:2272] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:05.878597 11368 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:05.880668 12068 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.917439 12987 ts_tablet_manager.cc:1916] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:05.927742 11443 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:05.934096 12969 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:05.930171 11268 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.939972 12065 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:15:05.942173 12969 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:05.948815 12885 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.953438 12818 raft_consensus.cc:1064] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:05.955492 12818 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:05.956512 11441 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.959599 12066 raft_consensus.cc:2955] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.962688 11436 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:15:05.964052 11367 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:15:05.966697 12065 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:15:05.967876 12818 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:05.970984 12970 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:05.975539 12955 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:05.986927 12971 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.995033 11368 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:05.998800 12987 ts_tablet_manager.cc:1929] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:15:05.999305 12987 log.cc:1199] T e5b7e70b18df44b6b0e248fbc3664d96 P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/e5b7e70b18df44b6b0e248fbc3664d96
I20260709 13:15:06.005230 11441 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.005509 12065 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.041340 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8a142ac56caa42a08e0a838dc87ac3b3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.074769 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet e5b7e70b18df44b6b0e248fbc3664d96 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:06.130834 11262 catalog_manager.cc:5697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 3 to 4, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:06.133740 11262 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 2 to 5, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.159386 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a142ac56caa42a08e0a838dc87ac3b3 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36642
I20260709 13:15:06.194062 12989 tablet_replica.cc:333] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:15:06.198966 11516 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.211551 11511 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.225245 12066 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:06.227865 12955 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:06.228749 12989 raft_consensus.cc:2243] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:06.230191 12989 raft_consensus.cc:2272] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:06.232153 11370 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:06.234570 12955 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:06.238368 12989 ts_tablet_manager.cc:1916] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:06.245426 12971 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.253456 12065 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:15:06.254390 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 47307.4us
I20260709 13:15:06.268491 12068 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:06.270649 11441 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:06.270670 12818 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:06.274027 12818 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:06.276938 12989 ts_tablet_manager.cc:1929] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:15:06.277653 12989 log.cc:1199] T 8a142ac56caa42a08e0a838dc87ac3b3 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/8a142ac56caa42a08e0a838dc87ac3b3
I20260709 13:15:06.281231 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 8a142ac56caa42a08e0a838dc87ac3b3 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:06.289048 12069 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.286603 12970 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.302452 11367 raft_consensus.cc:2955] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.309558 11266 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 4 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.320429 11436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:15:06.321247 11436 raft_consensus.cc:606] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:06.324859 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:15:06.325544 11516 raft_consensus.cc:606] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:15:06.327234 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:06.329670 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:06.332814 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b01b3301ee794d99bcb4feb672ac699f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.334599 11252 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:06.334652 11268 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index 4 to 5, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 2 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.320351 11441 raft_consensus.cc:2955] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:06.350615 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a992bbbf8dd4ccbb70ff8d65e31edae with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37074
I20260709 13:15:06.351925 12987 tablet_replica.cc:333] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:15:06.353149 12987 raft_consensus.cc:2243] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:06.355098 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet b01b3301ee794d99bcb4feb672ac699f with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51468
I20260709 13:15:06.356362 12987 raft_consensus.cc:2272] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:06.361455 12987 ts_tablet_manager.cc:1916] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:06.362468 12955 raft_consensus.cc:993] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:06.365201 12955 raft_consensus.cc:1081] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:06.367170 12065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:15:06.367802 12065 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:06.368147 12065 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:06.373750 12990 tablet_replica.cc:333] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:15:06.374936 12990 raft_consensus.cc:2243] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:06.375681 12065 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.375715 12990 raft_consensus.cc:2272] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:06.378279 12065 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:06.380950 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:06.381875 11367 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:06.378890 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:06.382874 11516 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:06.383190 11516 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:06.384539 11516 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.385772 11516 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:06.389195 11367 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:06.390595 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:06.391420 12987 ts_tablet_manager.cc:1929] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 13:15:06.391621 12986 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:06.391940 12987 log.cc:1199] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/1a992bbbf8dd4ccbb70ff8d65e31edae
I20260709 13:15:06.392252 12986 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:06.393478 11516 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:06.393198 12986 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.394098 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 1a992bbbf8dd4ccbb70ff8d65e31edae (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:06.394476 12990 ts_tablet_manager.cc:1916] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:06.404240 11268 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:06.406714 12990 ts_tablet_manager.cc:1929] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:15:06.408254 12990 log.cc:1199] T b01b3301ee794d99bcb4feb672ac699f P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/b01b3301ee794d99bcb4feb672ac699f
I20260709 13:15:06.410142 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet b01b3301ee794d99bcb4feb672ac699f (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:15:06.756366 12922 raft_consensus.cc:993] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:06.756837 12959 raft_consensus.cc:1081] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:15:06.758180 11369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37092
I20260709 13:15:06.758570 11369 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:06.758780 11369 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:06.763358 11369 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.764751 11369 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:06.765539 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:06.765676 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:06.766083 11436 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:06.766153 12068 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:06.766342 11436 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:06.767063 11436 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.768005 11436 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:06.772486 12068 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:06.773748 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:06.774675 12953 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:06.774916 11436 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:06.775153 12953 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:06.776072 12953 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:06.786047 11268 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:06.873821 11511 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:06.874851 12911 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:06.884193 11369 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:06.885753 12911 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:07.283206 11436 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:07.284389 12969 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:07.293952 12068 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:07.295205 13000 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:07.330782 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:07.426709 11370 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:07.432034 11516 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:15:07.432466 11334 consensus_peers.cc:597] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 857937fa61dc400a81345944f0a14511. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.433295 11443 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:07.433318 13000 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:07.434698 12969 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:07.441980 12999 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.443425 11511 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.443591 11443 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.459071 11441 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:07.461469 11263 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:07.464877 11516 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:15:07.465178 11409 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a5787304dfe4697baeaeb367ee7dda3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.466216 12959 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:07.476756 11369 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:07.478819 12959 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:07.492017 12922 raft_consensus.cc:2955] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.493243 11516 raft_consensus.cc:2955] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.503324 11369 raft_consensus.cc:2955] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.520175 11369 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:07.528126 11263 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:07.536036 11334 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 54c0e900e01b4e5aa4cf2fa0acbf6e86. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.537477 11516 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:07.539383 12999 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:07.542901 11436 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:15:07.545135 12969 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:07.564095 13000 raft_consensus.cc:2955] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.566036 11443 raft_consensus.cc:2955] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.566036 11516 raft_consensus.cc:2955] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.581858 11436 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:07.584363 11266 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 3, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:07.598384 11511 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:07.600309 12922 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:15:07.604089 11409 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7d370e878a484e1aaaa0df1466da6f24. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:07.610401 11370 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:15:07.611888 13012 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:07.612987 12959 raft_consensus.cc:2955] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.614445 11516 raft_consensus.cc:2955] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.617429 11370 raft_consensus.cc:2955] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.631562 11263 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index -1 to 2, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 1 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:07.897259 13021 ts_tablet_manager.cc:933] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:07.898947 13021 tablet_copy_client.cc:323] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:15:07.900946 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 7a5787304dfe4697baeaeb367ee7dda3 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:15:07.901417 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 7a5787304dfe4697baeaeb367ee7dda3 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-7a5787304dfe4697baeaeb367ee7dda3
I20260709 13:15:07.907729 11455 tablet_copy_source_session.cc:215] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:07.911093 13021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5787304dfe4697baeaeb367ee7dda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:07.925079 13021 tablet_copy_client.cc:806] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:07.925706 13021 tablet_copy_client.cc:670] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:07.930154 13021 tablet_copy_client.cc:538] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:07.937357 13021 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:07.958325 13021 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:07.959211 13021 tablet_bootstrap.cc:492] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:07.959954 13021 ts_tablet_manager.cc:1403] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 13:15:07.962229 13021 raft_consensus.cc:359] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:07.962957 13021 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:07.963665 13021 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:07.973124 13021 ts_tablet_manager.cc:1434] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.004s
I20260709 13:15:07.976807 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-7a5787304dfe4697baeaeb367ee7dda3 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:15:07.977252 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-7a5787304dfe4697baeaeb367ee7dda3 on tablet 7a5787304dfe4697baeaeb367ee7dda3 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:08.010712 13021 ts_tablet_manager.cc:933] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:08.012410 13021 tablet_copy_client.cc:323] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:15:08.014098 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 857937fa61dc400a81345944f0a14511 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:15:08.014544 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 857937fa61dc400a81345944f0a14511 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-857937fa61dc400a81345944f0a14511
I20260709 13:15:08.020064 11380 tablet_copy_source_session.cc:215] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.022950 13021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 857937fa61dc400a81345944f0a14511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.039669 13021 tablet_copy_client.cc:806] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.040225 13021 tablet_copy_client.cc:670] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.056902 13021 tablet_copy_client.cc:538] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.066632 13021 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:08.098379 13021 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:08.099392 13021 tablet_bootstrap.cc:492] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:08.100306 13021 ts_tablet_manager.cc:1403] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.034s	user 0.030s	sys 0.004s
I20260709 13:15:08.102466 13021 raft_consensus.cc:359] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:08.103055 13021 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:08.103799 13021 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:08.106851 13021 ts_tablet_manager.cc:1434] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:15:08.108731 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-857937fa61dc400a81345944f0a14511 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:15:08.109163 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-857937fa61dc400a81345944f0a14511 on tablet 857937fa61dc400a81345944f0a14511 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:08.117571 13021 ts_tablet_manager.cc:933] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:08.121686 13021 tablet_copy_client.cc:323] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:15:08.123168 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 7d370e878a484e1aaaa0df1466da6f24 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:15:08.123651 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 7d370e878a484e1aaaa0df1466da6f24 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-7d370e878a484e1aaaa0df1466da6f24
I20260709 13:15:08.129482 13027 ts_tablet_manager.cc:933] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:08.130733 11455 tablet_copy_source_session.cc:215] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.131278 13027 tablet_copy_client.cc:323] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:15:08.133136 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:15:08.133673 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-54c0e900e01b4e5aa4cf2fa0acbf6e86
I20260709 13:15:08.133903 13021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d370e878a484e1aaaa0df1466da6f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.143324 11380 tablet_copy_source_session.cc:215] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:08.149128 13027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:08.155886 13021 tablet_copy_client.cc:806] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.157756 13021 tablet_copy_client.cc:670] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.161995 13021 tablet_copy_client.cc:538] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.171203 13021 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:08.175652 13027 tablet_copy_client.cc:806] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:08.176359 13027 tablet_copy_client.cc:670] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:08.184186 13027 tablet_copy_client.cc:538] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:08.193733 13027 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:15:08.196004 13021 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:08.197306 13021 tablet_bootstrap.cc:492] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:08.197988 13021 ts_tablet_manager.cc:1403] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.008s
I20260709 13:15:08.200590 13021 raft_consensus.cc:359] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:08.201372 13021 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:08.201954 13021 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:08.207814 13021 ts_tablet_manager.cc:1434] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:15:08.209460 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-7d370e878a484e1aaaa0df1466da6f24 received from {username='slave'} at 127.0.0.1:41514
I20260709 13:15:08.209887 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-7d370e878a484e1aaaa0df1466da6f24 on tablet 7d370e878a484e1aaaa0df1466da6f24 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:15:08.219462 13027 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:08.220417 13027 tablet_bootstrap.cc:492] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:15:08.221134 13027 ts_tablet_manager.cc:1403] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.005s
I20260709 13:15:08.225152 13027 raft_consensus.cc:359] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:08.226191 13027 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:15:08.226876 13027 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:08.230472 13027 ts_tablet_manager.cc:1434] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:15:08.232213 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-54c0e900e01b4e5aa4cf2fa0acbf6e86 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:15:08.232543 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-54c0e900e01b4e5aa4cf2fa0acbf6e86 on tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 with peer b1906ab37ba541f493936685a808c8a9
W20260709 13:15:08.420019 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 74726.8us
I20260709 13:15:08.437007 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:09.527446 11391 debug-util.cc:398] Leaking SignalData structure 0x7b08001c5ac0 after lost signal to thread 10260
I20260709 13:15:09.707813 13036 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:15:09.707890 11391 debug-util.cc:398] Leaking SignalData structure 0x7b08001961a0 after lost signal to thread 11302
I20260709 13:15:09.711961 13039 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.712508 13039 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.714529 13039 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.715049 13035 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.709877 13036 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:09.717036 13041 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.716814 13035 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.717588 13041 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.718796 13036 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.718953 13035 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.719861 13041 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.722112 13040 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.722750 13040 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.724977 13040 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.714989 13039 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.720764 13042 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.736851 13038 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.737401 13038 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.761751 13039 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:09.760654 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44444.7us
I20260709 13:15:09.732496 13041 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.769686 13041 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:09.771883 13041 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.782907 13039 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:09.760244 11391 debug-util.cc:398] Leaking SignalData structure 0x7b08000cce80 after lost signal to thread 11394
I20260709 13:15:09.765261 12953 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.785262 12953 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.759552 13037 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.734499 13040 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.769011 13038 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.790840 13035 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.737874 13042 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.792622 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:09.786404 13037 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.790853 13039 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.802508 13043 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.803089 13043 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:09.804031 12955 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.802747 13039 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.804497 12955 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.805128 13043 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.805497 13037 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.806210 12955 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.806900 13039 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.807493 13039 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.808007 13039 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.808328 13042 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.792263 13035 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.810518 13035 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.811285 12953 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.812062 13039 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.791324 13040 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:09.816864 12955 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.816180 13044 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.817435 12955 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.793246 13036 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.839064 13045 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.838914 13036 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.839188 13047 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.843120 12955 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.837080 13039 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.844159 13039 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.846700 13039 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.828625 13044 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.855319 13055 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.858546 13050 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.861946 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:09.851261 13017 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.861191 13044 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.869781 13046 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.869853 13044 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.850871 13047 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.870396 13046 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.873270 13042 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.870599 13044 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.841091 13045 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.856009 13055 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.873585 13017 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.882966 12955 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.904192 13043 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.859210 13050 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.921819 13049 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.902845 13047 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.905267 13040 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.921909 13043 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.882890 13039 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.887598 13037 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.925585 13043 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.923889 12955 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.882211 13046 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.928431 13046 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.924568 13039 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.928922 13046 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.926474 13037 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:09.929257 13055 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.934787 12959 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.922413 13049 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.952313 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:09.956354 13052 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.955226 12959 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.962060 12959 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.935369 13056 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.963415 12955 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.964929 12953 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.965546 13050 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.964982 13046 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.966476 13036 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.967713 13039 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.965517 12953 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.954559 13055 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.887542 13042 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.981181 13036 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.981562 13045 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.981916 12953 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.982031 13039 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.982402 12953 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.982225 13056 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.984591 12953 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:09.985675 13036 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.987090 13056 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.987461 12953 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:09.983855 13035 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.982609 13039 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.956979 13052 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.990899 13035 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.991235 12922 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.970121 13050 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.980432 13046 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:09.993534 13039 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.993230 12922 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:09.998487 13037 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:09.995311 13052 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.995800 12953 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.003295 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:09.999033 13037 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.004433 13037 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.005352 12953 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.006325 13037 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:09.983096 13055 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:09.968526 12959 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:09.984301 13045 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.009232 13036 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.009657 12922 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.008730 13017 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.013036 13055 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.003664 13043 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.013932 13043 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.014619 13036 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.015139 13036 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.016012 13043 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.016707 13042 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.017777 13044 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.019454 13035 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.020326 13039 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.020874 13039 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.022200 13000 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.022699 13000 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.009249 13045 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.024413 13036 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.024600 13000 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.025283 13045 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.025883 12922 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.026150 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.026262 12065 raft_consensus.cc:1217] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:09.999450 13050 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:10.026803 11367 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 1.
I20260709 13:15:10.028060 12033 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:10.028362 13050 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.030154 13045 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.030458 13050 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.030836 13051 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.031487 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.032127 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.032379 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.033567 12911 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:09.994239 13056 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:09.994379 13049 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.043653 12911 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.045396 12959 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.043140 11437 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:15:10.063488 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.064175 11443 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:10.011623 12953 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.037133 13045 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.037498 13055 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.082295 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.090024 11367 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 1.
I20260709 13:15:10.082356 12911 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.091208 12911 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.062517 13052 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.026368 12922 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.091682 12911 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.095799 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.080032 13039 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.008718 13046 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.097819 12955 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.098405 13047 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.098979 13043 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.104281 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.099704 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.102197 12955 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:10.105644 11443 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:15:10.099960 13017 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.100170 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.063284 13056 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.066658 13051 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.104034 13052 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.098944 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.086859 13045 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:10.108606 13047 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.111626 13017 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.104620 13043 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.116600 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.131367 12911 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.136128 12955 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.082417 13050 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.140558 11441 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:10.144778 12068 raft_consensus.cc:1217] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:15:10.147441 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:10.149320 12911 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.149677 12911 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.147789 11516 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:10.150043 12911 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:10.136906 13012 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.106940 13055 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.155154 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.159209 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.159830 11511 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:10.160867 11268 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39438 (request call id 356) took 1447 ms. Trace:
I20260709 13:15:10.161289 11268 rpcz_store.cc:276] 0709 13:15:08.713743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.713911 (+   168us) service_pool.cc:224] Handling call
0709 13:15:10.160791 (+1446880us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3863,"reported_tablets":1}
I20260709 13:15:10.162297 13042 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.162829 13042 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.163144 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.164813 13042 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.287516 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.271332 13057 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.303061 13017 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.306139 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.306851 12068 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:10.303289 13057 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.322283 13047 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.323886 12922 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.326064 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.272682 13058 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.328018 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.329680 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.330307 12065 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:10.272149 13061 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.165427 13012 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.351900 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.353318 13012 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.355748 12911 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.357862 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.327865 13058 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.360175 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.360965 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:10.360225 12911 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.361526 11367 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:10.362093 13058 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.353899 13061 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.362699 13065 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.312335 12959 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.365343 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.366268 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.366837 11368 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 1.
I20260709 13:15:10.367892 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.368453 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.369014 11444 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:10.369233 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.370338 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.370709 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.370905 11511 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:10.371294 11437 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:10.378096 13037 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.379933 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.381856 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.382184 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.380584 11437 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:10.384228 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.383428 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.387383 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.385582 11514 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:10.388628 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.373265 13047 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.371804 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.398643 11367 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:10.363209 13065 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.402834 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.391817 13043 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.407173 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.407486 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.407848 11367 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:10.380244 13037 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.411690 12065 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:10.408993 13058 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.406069 13062 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.408733 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.414065 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.414284 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.414764 12066 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 2: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:10.404170 11513 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:10.418262 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.418484 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.410485 12959 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.410516 13064 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.422127 12955 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.404364 13047 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.422708 12955 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.413142 13041 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.424017 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.424571 13051 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.419598 13060 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.404814 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.419456 13066 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.427246 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.427014 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.415598 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.430589 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.432832 12068 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:10.433712 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.416229 13062 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.484500 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.484879 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.486531 12955 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.488157 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.429613 13043 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.488070 11335 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.420562 13058 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:10.494971 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 81917.5us
I20260709 13:15:10.496300 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.428989 13064 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.496946 11445 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:10.429534 12959 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.499805 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.431383 13041 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.497876 13034 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:10.486410 11514 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:10.504688 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.494657 13061 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.484671 13066 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.503614 12953 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:10.516300 12953 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.507432 13034 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.507536 11438 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:10.510257 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.517237 11511 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:10.507601 13038 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.511210 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.511725 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.429440 13060 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.513994 13062 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.513311 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.516191 13037 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.507167 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.515436 13047 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.515525 13041 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.512677 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.530553 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.512641 13071 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.517501 12953 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:10.498245 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.499337 13043 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.524333 13038 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.530352 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.503788 13069 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.532020 11368 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
W20260709 13:15:10.535161 11333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.519s	user 0.033s	sys 0.029s
I20260709 13:15:10.517625 13034 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:10.542783 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.633519 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:10.619989 11333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.615s	user 0.034s	sys 0.029s
I20260709 13:15:10.528947 13049 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.635113 13049 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.635792 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.634133 11519 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:10.636354 11517 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:10.635085 11408 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.637225 13049 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.527762 13067 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.637617 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.531680 13042 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.637734 13049 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.638260 13042 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.638520 13049 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.549052 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:10.546002 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.556530 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.640247 13043 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.640628 13049 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.549722 13052 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.555186 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.641857 13060 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.642093 12067 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:10.642570 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.623257 13071 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.538156 12986 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.628578 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.648234 13071 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.650625 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:10.580641 11391 debug-util.cc:398] Leaking SignalData structure 0x7b08001fc460 after lost signal to thread 11469
I20260709 13:15:10.575788 13055 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.557253 13046 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.625365 13065 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.626221 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.627435 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.628468 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.631588 13066 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.631729 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.627345 13053 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.637717 13051 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:10.655871 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.656203 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.689375 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.557750 13050 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.694566 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.624095 13056 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:10.697686 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 70348.7us
I20260709 13:15:10.698422 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.640259 13042 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.646293 13017 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.637264 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:10.708722 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 59259.4us
I20260709 13:15:10.654124 13060 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.649003 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.653577 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.673506 13043 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.714711 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.716197 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.718238 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.702641 13051 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.706588 13066 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:10.702031 13039 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:10.709955 13045 consensus_meta.cc:287] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.000s
I20260709 13:15:10.722375 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.711376 13017 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.665280 12986 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.711771 13060 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.767853 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.719280 13043 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.716879 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:10.776396 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59572.1us
W20260709 13:15:10.723610 11391 debug-util.cc:398] Leaking SignalData structure 0x7b08003525c0 after lost signal to thread 11543
I20260709 13:15:10.771246 11511 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:10.723760 13051 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:10.778780 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 80735.3us
I20260709 13:15:10.777027 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.723927 13066 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.777412 13060 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:10.649601 13069 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.787081 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.786583 11516 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:10.785038 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:10.788134 11370 raft_consensus.cc:606] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:10.793948 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 84644.8us
I20260709 13:15:10.797302 13038 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.798035 13033 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:10.798410 13033 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:10.798739 13033 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:10.780069 13037 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.786548 13066 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:10.783449 13047 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:10.803287 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:10.803736 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:10.806591 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:10.807449 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 99749.3us
I20260709 13:15:10.807561 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.808158 13078 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.808552 13078 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:10.808912 13078 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:10.810289 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.811761 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.813133 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.813774 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:10.778270 13045 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.637982 13067 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.801473 13037 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.709039 13053 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:10.823930 13037 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:10.824012 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.824506 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:15:10.787910 13039 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.826071 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:15:10.816581 11391 debug-util.cc:398] Leaking SignalData structure 0x7b0800309ca0 after lost signal to thread 12093
I20260709 13:15:10.806116 13047 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:10.834329 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:10.814110 13069 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:10.820785 13071 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:10.836586 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.872371 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.082924 12959 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.093077 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:11.104548 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 301297us
I20260709 13:15:10.844208 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.105506 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:10.883854 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.107780 11334 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:11.108289 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:10.936254 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 298723us
I20260709 13:15:11.110580 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:11.111253 12959 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.111590 12959 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:11.111908 12959 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.113194 11408 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:11.114334 11408 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:11.105994 12065 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:15:10.881317 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:10.872116 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.124648 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.108767 13035 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.880751 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.128500 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:11.129191 12911 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:11.128928 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 357004us
I20260709 13:15:11.130717 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.131314 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:11.071242 12953 consensus_meta.cc:287] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.000s
I20260709 13:15:11.126183 13035 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:11.154186 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:10.963973 13043 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:10.834148 13047 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.153893 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:10.881186 13039 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.933696 12986 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.153905 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.081750 13017 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.152459 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.134770 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.155808 11334 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:11.158118 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:10.889514 12064 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:11.126813 13052 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.156702 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:10.954722 13072 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.880268 13067 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:10.839818 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.179466 13072 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:10.951483 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.157250 13035 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:10.981752 13058 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.163048 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.181192 11364 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:15:11.163164 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.152356 13045 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:11.166749 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:11.152383 13046 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:11.205623 13046 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:11.205751 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.206108 13046 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.125162 13071 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:11.215883 13071 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:11.152235 13049 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:11.166275 12066 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
W20260709 13:15:11.217491 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 60888.1us
I20260709 13:15:11.083088 12062 raft_consensus.cc:1217] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:15:11.131965 13040 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.071902 13066 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:10.881645 11365 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:11.152081 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.169739 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.244293 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.245239 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.174777 13053 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:11.174942 12069 raft_consensus.cc:1217] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:15:11.158658 13079 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.167299 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.178915 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.179898 13072 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.166468 11370 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.182349 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.214807 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:11.206933 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.215389 13057 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:11.217756 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 51088.5us
I20260709 13:15:11.217933 12911 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.881081 13078 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.257398 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.257612 12911 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:11.258024 12911 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.259018 13078 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.167299 13039 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:10.939291 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.226780 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:11.173611 12986 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.279476 13039 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:11.279843 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.280968 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.173871 13052 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:11.245769 13041 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:11.288314 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.289719 12911 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:11.241250 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 60853.5us
I20260709 13:15:11.241657 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:11.246510 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.249886 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.250036 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.250293 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.297415 11517 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:11.250453 13079 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:11.252971 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 120103us
I20260709 13:15:11.298326 11408 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:11.298753 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.225829 13049 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:11.226132 13076 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:11.259192 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:11.274963 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 50213.5us
I20260709 13:15:11.273506 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.261962 11514 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:11.278530 13051 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.279961 13039 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.153165 12953 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.299443 11369 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:11.303241 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.303681 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:11.299885 11408 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:11.259559 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:11.305570 12911 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.305634 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:15:11.306466 13060 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.522s	user 0.005s	sys 0.012s
I20260709 13:15:11.297463 13044 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:11.332468 13054 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.307003 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.332971 13075 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.333446 13075 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:11.303593 13049 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:11.331910 11408 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:11.334796 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:11.333834 13075 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.335819 11408 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:11.341874 11370 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:11.342640 11368 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.296993 13052 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.344789 12959 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.346788 11436 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.301630 13079 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.303723 13057 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.346006 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:11.352192 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 148714us
I20260709 13:15:11.354238 12033 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:11.304844 13076 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:11.356410 11366 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.356983 12911 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:11.357452 12911 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:11.357801 12959 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.364632 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:11.366549 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.360688 12911 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.343663 13044 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:11.345656 13058 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:11.368258 11335 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:11.375628 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.347951 13061 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:11.352661 13054 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:11.309903 13041 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.310644 11510 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:11.335316 13046 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:11.332477 13060 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.393632 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:11.394054 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 53345.2us
I20260709 13:15:11.355327 13057 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:11.391115 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.361693 13076 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.394114 11515 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:11.395589 11408 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:11.351377 13035 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.382951 13058 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:11.396371 13057 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.383287 13061 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:11.389892 13054 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.404677 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.414974 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:11.415884 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.415565 13044 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.415597 12068 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.399921 13050 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.419550 13080 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:11.402812 13058 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:11.402621 13061 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.424598 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:11.430945 12959 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:11.432660 12959 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:11.417659 13034 consensus_meta.cc:287] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.713s	user 0.001s	sys 0.015s
I20260709 13:15:11.433115 12959 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.437438 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:11.421425 11438 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.438009 11364 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.425446 13066 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.410920 13082 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.422183 13050 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:11.434737 13080 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:11.447214 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.447916 13035 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.448514 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.449533 13080 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.417500 13064 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:11.451197 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 67478.7us
I20260709 13:15:11.454512 12068 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:11.455570 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.450312 13050 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.437697 13034 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.476188 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.455552 13082 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:11.470767 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.484287 11363 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.538563 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.499334 11364 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:11.518185 12959 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.532827 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.560688 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:11.524504 13037 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.700s	user 0.005s	sys 0.006s
I20260709 13:15:11.565655 12959 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.568234 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.485170 13082 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.551072 12061 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.493484 11361 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.534754 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1380) took 1023 ms. Trace:
I20260709 13:15:11.485468 13041 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.486145 12063 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.574658 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:11.576184 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.592471 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.579062 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.580464 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.598747 11516 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:11.580883 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.519557 13055 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:11.528045 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.487246 12953 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.519891 13046 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.567816 13035 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:15:11.577805 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.521715 13064 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:11.594954 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:11.568353 13045 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:11.598066 13039 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.629650 13045 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:11.630515 13045 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.631043 13039 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.607043 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.601528 13073 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:15:11.608144 13081 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:15:11.610695 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.614275 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.639786 12911 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.497463 13049 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.608611 13058 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:11.621148 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 227156us
I20260709 13:15:11.644724 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.568971 13071 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.528580 13060 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.622670 12062 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.645619 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.647090 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.647933 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.649917 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.637878 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:11.628058 13088 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:15:11.617328 13055 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:11.629048 13033 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.003s
I20260709 13:15:11.633777 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:11.634462 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.635351 13044 consensus_queue.cc:1059] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:11.637646 13064 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.629101 13035 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:11.640774 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.573720 13037 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.626924 13082 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.650826 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.651708 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:11.652829 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 179558us
I20260709 13:15:11.586134 11365 rpcz_store.cc:276] 0709 13:15:10.511534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.536750 (+ 25216us) service_pool.cc:224] Handling call
0709 13:15:11.528768 (+992018us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54001e69d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:11.528887 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293677312}
I20260709 13:15:11.655233 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.657202 12067 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:11.641995 13046 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.631458 13076 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.637966 13054 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.676901 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.666814 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.678960 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.656010 13034 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.684418 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.686712 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.687877 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.688390 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.688735 11370 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.690920 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:11.663774 13066 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.694012 13074 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.694897 13054 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.695561 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.706882 13066 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:11.709642 13066 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.710090 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:11.696170 13058 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.660604 13035 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:11.669013 13065 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.719569 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:11.687824 11518 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:11.693948 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.701040 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.708902 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:11.698153 13079 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.694845 13037 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:11.729842 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.706020 13041 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.733089 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.731654 13043 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:11.725347 13047 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.552s	user 0.006s	sys 0.004s
I20260709 13:15:11.734876 13086 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:15:11.751307 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.753844 13091 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.736305 13057 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:11.713886 13034 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:11.742733 11436 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.742794 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.734205 13049 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.746078 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.659251 13033 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.693003 13060 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:15:11.744473 13062 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.747606 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:11.800235 11443 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:11.800880 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 50569.8us
W20260709 13:15:11.801889 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 55876.3us
I20260709 13:15:11.752319 13076 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.805728 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.808593 11444 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:11.801723 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 133786us
I20260709 13:15:11.810393 13055 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.810902 13055 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:11.811122 12911 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:11.811553 12911 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:11.811995 13062 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.812377 12911 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.813689 12911 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.814641 13053 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:11.815012 13053 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:11.814652 11437 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.816403 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.817515 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.806314 13058 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:11.806541 13043 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.822516 13053 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.824517 13090 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:11.831014 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 89727us
I20260709 13:15:11.749890 13047 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.832084 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:11.830497 13061 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.802629 13034 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:11.846002 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.849323 11363 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.836737 13053 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.751479 13072 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:11.822705 13041 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.836539 13090 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:11.811198 13055 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:11.855760 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:11.846839 13043 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.856772 11440 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.858048 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:11.837364 13058 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:11.843019 13071 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:11.863965 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:11.864125 13079 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.864485 11510 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.827020 13050 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.860174 13090 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.886865 13080 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.865649 13057 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.867116 11519 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.822705 13076 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
W20260709 13:15:11.864717 13075 consensus_meta.cc:287] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.515s	user 0.004s	sys 0.005s
I20260709 13:15:11.869515 13082 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.871848 11361 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:11.839823 13066 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.873766 13058 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.822360 13089 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:15:11.875245 13034 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.904284 11445 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.862978 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.877797 13061 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.907159 11437 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:11.866751 13052 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.001s
I20260709 13:15:11.885221 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.908495 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:15:11.883178 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 206340us
W20260709 13:15:11.909912 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47028.2us
I20260709 13:15:11.910670 12032 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.802428 13035 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.860764 13064 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.914605 13058 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.912858 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1549) took 1262 ms. Trace:
I20260709 13:15:11.897210 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.883615 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.917613 11369 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:11.917336 13094 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:15:11.914463 13096 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.920001 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:11.920593 13051 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:11.915195 11519 rpcz_store.cc:276] 0709 13:15:10.649593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.069880 (+420287us) service_pool.cc:224] Handling call
0709 13:15:11.070223 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.891983 (+821760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.911450 (+ 19467us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54002c1d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:11.911591 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832291712}
I20260709 13:15:11.912904 13067 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:11.917821 13076 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:11.921734 13076 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:11.920717 11367 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:11.923187 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.924265 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.925845 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.926131 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.931682 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:11.933699 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.934965 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.933625 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.942551 11515 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 2: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:11.942983 11442 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.935767 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.936537 13055 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.940912 13050 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.939817 13066 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.922547 13076 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:11.940956 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:11.897629 13075 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.947489 13076 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:11.948081 11365 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:11.934880 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.944940 13049 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:11.945436 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.945533 12953 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:11.950654 11370 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:11.946280 13055 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:11.977553 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:11.944428 12911 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:15:11.955742 11333 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.956981 13075 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:11.958114 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:11.933157 13067 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.981937 11333 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.979753 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.914009 13052 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:11.984141 11333 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:11.936118 13096 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.923199 13081 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:15:11.980829 11438 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:11.988106 12911 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:15:11.988304 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:11.977293 13073 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:11.980404 12953 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:11.982162 11409 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 1.569s (SENT)
I20260709 13:15:12.022656 11441 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.027165 11333 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:11.984846 13100 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.029201 13080 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:12.029757 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:11.985368 13040 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:11.986008 13079 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:12.011538 13067 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.034189 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.016700 12953 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.035737 13075 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:12.036275 13075 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:12.036463 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.036755 13080 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.037914 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.038524 13075 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.046029 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.048032 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.049155 11370 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.049535 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.051352 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.055677 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.061286 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.047729 13080 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.035741 13099 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:12.067447 13099 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:12.067829 13099 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.068020 13052 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:12.062477 11333 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.031473 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:12.064064 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 81930.3us
I20260709 13:15:12.064594 13084 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:15:12.065297 13071 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:12.065768 13062 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:12.065950 13037 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:12.043013 13090 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:12.059921 13076 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:12.061975 13081 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:12.044709 13075 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:12.069525 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 78861.7us
I20260709 13:15:12.071787 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.063818 13040 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:12.073731 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.072613 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.075632 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.063766 13041 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.070927 13080 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:12.074496 13081 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:12.068580 13069 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:12.073719 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 94711.3us
I20260709 13:15:12.077840 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.080041 11511 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.074644 13076 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:12.078540 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.087469 13103 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.089399 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.091642 11366 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:12.091322 11511 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:12.093924 11516 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.095955 13105 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:12.097070 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.098156 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:12.099141 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:12.100349 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.101537 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.101820 13084 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.102847 13084 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.103446 13084 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:12.103771 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.100394 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.105074 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:12.106696 11438 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.106678 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.107559 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.108031 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.079679 13081 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.110524 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.121870 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.124670 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.126016 13101 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:12.126432 13101 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:12.126717 11516 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.127383 13101 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.128822 13101 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:12.131279 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.132063 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.091590 11514 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.147166 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.108361 13105 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:15:12.092841 13076 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.151024 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.146999 11513 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.153548 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.156625 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.123643 13100 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.076823 13077 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:15:12.131570 13067 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.159446 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.158978 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.173610 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.176393 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.174856 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.177732 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.155185 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.177991 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.155006 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.178530 13107 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.181354 13044 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:12.186476 13075 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:12.164407 13049 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.179757 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.166134 13064 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.189502 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.160357 13082 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:12.178915 13052 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.159315 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.192759 13050 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.205833 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.210640 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.228726 13085 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:12.249603 13085 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:12.210444 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.184258 13105 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.231761 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:12.241199 12986 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:12.243180 13047 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:12.257688 11439 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:12.258157 13065 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:12.255748 11361 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:12.259029 13065 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:12.214001 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.259963 13065 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.211112 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1551) took 1507 ms. Trace:
I20260709 13:15:12.206444 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.224584 13033 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:12.246182 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:12.222416 13035 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.202855 13091 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:12.246240 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.257164 12986 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:12.250535 13085 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:12.260913 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.213011 13043 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.252238 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.259248 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.201385 12953 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:12.353982 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 90339.3us
I20260709 13:15:12.299422 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:12.354230 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 260568us
I20260709 13:15:12.355413 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:12.265118 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:12.357020 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 91999.5us
I20260709 13:15:12.357015 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.296782 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.360466 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:12.359196 11334 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.261255 13065 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.366086 11334 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.365852 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.271179 13107 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:12.265928 13052 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:12.296037 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.367838 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.310271 13090 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:12.361495 12064 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:12.295385 13072 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:12.359043 13094 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.367466 13055 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:12.380283 13102 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:12.380769 11513 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:12.372365 13046 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.363667 13075 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:15:12.370543 13107 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:12.383275 13052 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:12.383908 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.368957 13077 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.363029 13040 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.370498 13066 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.264299 13064 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:15:12.369726 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.301024 13044 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:15:12.359231 13091 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:12.371930 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:12.474860 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 102992us
I20260709 13:15:12.462353 11361 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.413831 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:12.485527 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.485963 11442 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:12.414283 11437 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.414665 11443 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:12.410528 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.298632 13098 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.419759 13093 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:15:12.460968 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.401739 13073 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:12.459335 12953 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:12.263804 11518 rpcz_store.cc:276] 0709 13:15:10.703658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.982898 (+279240us) service_pool.cc:224] Handling call
0709 13:15:11.154975 (+172077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.721718 (+566743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.210856 (+489138us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400064fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:12.211034 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156699904}
I20260709 13:15:12.409518 13055 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:12.411011 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.460539 13107 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.420542 13102 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:12.421480 13079 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:15:12.508137 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.516837 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.518023 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.476927 13091 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:12.514240 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.518877 12959 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.519658 12959 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.520231 12959 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.490865 13050 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.396632 13100 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:15:12.515198 13039 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.488682 13105 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:15:12.510766 13055 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:12.493403 13073 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.552649 13073 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:12.515620 13054 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.511749 13060 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:15:12.566779 13102 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.568495 13102 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.513603 13081 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.617286 11367 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.620206 11364 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:12.623682 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 443965us
I20260709 13:15:12.604224 13060 consensus_queue.cc:1243] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:12.624502 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.564553 13052 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:12.694943 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70506.3us
I20260709 13:15:12.695974 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.710462 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.711998 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.712647 11441 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.712425 11519 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.712193 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:12.719352 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.720939 11483 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.723742 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.699438 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.725541 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:12.726151 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.730921 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.740911 13111 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.710618 13041 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:12.701725 13110 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.746579 13112 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.706013 12969 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:12.706477 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.736303 11408 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:12.758720 12959 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:12.700707 13083 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:12.749580 13051 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:12.734658 13062 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.744261 13064 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.702802 12986 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.735721 13043 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.724056 13017 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:12.724838 13084 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:12.759508 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 389834us
I20260709 13:15:12.735442 13058 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:12.761052 12959 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:12.698096 13061 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:15:12.763875 13116 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.764114 12953 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:12.764065 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:12.728621 13098 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:15:12.749248 13111 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:12.767269 13076 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.596s	user 0.006s	sys 0.000s
I20260709 13:15:12.751941 13041 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:12.758893 12969 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:12.759932 13083 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:12.780084 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.780949 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.781384 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.768219 12060 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:12.787793 12959 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:12.797180 13038 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:12.801759 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.817165 11409 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.818058 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.819121 11409 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.823642 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.801985 13119 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.823853 11518 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.835717 11361 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:12.836786 13119 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.822005 13058 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:12.838209 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.811514 12986 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.839167 11361 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.834357 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:12.841775 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.843065 12069 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:12.843931 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:12.844493 12069 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
W20260709 13:15:12.803380 13099 consensus_meta.cc:287] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.000s
I20260709 13:15:12.789111 12953 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:12.849723 11361 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.868322 13087 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:12.850482 13114 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:15:12.851382 12068 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:12.789821 13111 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:12.868741 11368 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:12.845350 13042 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:12.871279 13042 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.873507 13042 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:12.793223 13076 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.855618 13069 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:12.825423 13061 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:12.801017 13098 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:12.922021 13076 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:12.801048 12969 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:12.922719 11365 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.800475 13038 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.922021 13098 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:12.920624 13073 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:15:12.834491 13056 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:12.854729 13101 consensus_meta.cc:287] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.725s	user 0.006s	sys 0.005s
I20260709 13:15:12.934964 13040 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:12.935487 13040 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:12.936470 13040 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:12.937305 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.824301 13060 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:12.814244 13081 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:12.848942 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.868757 13087 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:12.875558 13099 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.796763 13118 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:15:12.900585 11367 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.900632 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:12.962666 11369 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:12.901010 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:12.901739 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.900843 13046 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:12.963423 11370 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:12.908550 13041 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:12.911509 12953 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:12.912278 13111 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:12.916230 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 92663.7us
I20260709 13:15:12.920746 13033 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:12.924152 13098 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:12.932350 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.966943 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.939657 11333 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:12.942831 11511 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:12.857540 11362 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:12.944970 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:12.923581 13065 consensus_meta.cc:287] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.000s
I20260709 13:15:12.974084 11366 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:12.937655 13083 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:12.951864 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:12.924649 13073 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:12.958065 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:12.930105 13038 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:12.958942 12062 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:12.957952 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:12.973057 12953 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:12.963263 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 114418us
I20260709 13:15:12.937968 13040 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:12.935542 13101 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 13:15:13.011682 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 61043.9us
I20260709 13:15:12.966380 13072 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:13.013051 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:12.966396 13069 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:15:12.961180 13105 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:15:12.967571 13033 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:12.903676 13039 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:12.973428 13111 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:13.011979 11439 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:13.015102 12959 consensus_queue.cc:1175] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "01ae50c877d04ba687e4e06259ba5581" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:15:13.049239 13033 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.013979 13050 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:13.013767 13049 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:13.015633 13056 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:13.013336 13073 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:12.965602 13046 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:13.035029 13060 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:13.018911 13066 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:13.042397 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 96718us
I20260709 13:15:13.022204 13083 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.047922 11367 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.015897 13075 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:15:13.054745 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:12.961202 13107 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.057514 11437 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:13.040540 13098 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.035358 13123 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:13.020197 13037 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:15:13.045624 13039 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:13.065773 12060 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:13.066759 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.069433 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:13.059343 13049 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:13.057821 13050 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:13.082672 12066 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.081550 11440 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:13.083313 13123 consensus_queue.cc:1243] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:13.067435 13066 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:13.083735 11513 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:13.077433 13126 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:15:13.072913 13034 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:13.077664 13046 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.087837 13049 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.088428 13046 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.089177 13049 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.091917 13126 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:13.092162 13107 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.075925 13060 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:13.093963 13055 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.000s
I20260709 13:15:13.100355 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:13.083689 13038 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:15:13.102841 11334 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:13.086890 13075 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:13.106052 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:13.118863 11334 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:13.084352 13066 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.154341 13085 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.106106 13081 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:13.118312 13118 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:15:13.190033 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.100389 13055 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.114990 13039 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.194720 13039 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.153162 13093 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:13.199373 13109 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:13.199848 13109 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:13.120774 13091 consensus_meta.cc:287] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 13:15:13.106582 13105 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 13:15:13.168141 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.200604 13109 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.101642 13073 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.189925 13050 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.192821 13035 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:15:13.210602 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.190508 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.231320 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 194459us
I20260709 13:15:13.231190 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1575) took 1159 ms. Trace:
I20260709 13:15:13.155296 13038 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:13.213021 11263 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.238946 11519 rpcz_store.cc:276] 0709 13:15:12.072034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.075438 (+  3404us) service_pool.cc:224] Handling call
0709 13:15:12.712238 (+636800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.717219 (+  4981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.230933 (+513714us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540014bed0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:13.231118 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463895808}
W20260709 13:15:13.229583 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 154392us
I20260709 13:15:13.245751 13084 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:13.246170 13084 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:13.233546 13102 consensus_meta.cc:287] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.665s	user 0.006s	sys 0.006s
I20260709 13:15:13.228230 13066 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.244223 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:13.251106 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:13.215238 12986 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:13.252311 13098 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:13.246984 13084 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.256666 13129 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.254614 13089 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:13.202525 13060 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.231791 13091 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.286130 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:13.257937 13109 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.284276 11370 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.240787 13038 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:13.256265 12986 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:13.317481 11515 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:13.260710 13061 raft_consensus.cc:1064] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:13.302740 13122 raft_consensus.cc:1064] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:13.259526 13050 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.319131 13089 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:13.307327 13084 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:13.258337 13098 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:13.320169 13072 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:13.323796 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 246298us
W20260709 13:15:13.327663 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.331104 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:13.256305 13073 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.333314 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:13.332340 13071 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:13.329550 11510 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:13.352783 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.332031 13098 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.332576 13055 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.324977 13056 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:15:13.346544 13071 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:15:13.357774 13033 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:13.353464 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:13.342170 13017 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:15:13.354223 13111 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:15:13.335110 13072 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.332779 12986 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.297719 13129 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:13.419595 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 137815us
I20260709 13:15:13.433142 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:13.417404 13099 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.447523 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:13.352663 13038 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.452630 13129 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:13.441241 13089 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.441062 13131 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:13.449236 13055 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:13.533878 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:13.534746 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:13.534711 11369 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:13.455758 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:13.509227 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 331) took 1127 ms. Trace:
I20260709 13:15:13.498862 13115 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:13.539484 13102 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:13.540288 13061 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.479547 13091 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.537467 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:13.537526 13055 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.522804 13131 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:13.469388 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:13.533809 13133 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:15:13.538789 11365 rpcz_store.cc:276] 0709 13:15:12.381372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.834661 (+453289us) service_pool.cc:224] Handling call
0709 13:15:12.835000 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.923149 (+ 88149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.508968 (+585819us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b5400423e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:13.509158 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201306880}
I20260709 13:15:13.545095 12033 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:13.527971 13072 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.550415 13137 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:13.542426 13115 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:13.574704 13042 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:13.575325 13042 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.566677 13141 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:13.581943 11436 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.585637 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.582257 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.583243 11445 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:13.586395 11441 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:13.586380 11443 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:13.587921 11437 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.578819 13131 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.585860 13141 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.604197 13138 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:15:13.599675 13065 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:13.616644 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.602113 13078 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:13.619320 12959 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:13.605806 13141 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:13.604369 13143 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:15:13.623867 12959 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:13.624292 12959 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:13.625059 12959 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.626403 12959 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.629355 11438 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:13.634168 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 329) took 1296 ms. Trace:
I20260709 13:15:13.622763 13078 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.573451 13115 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.620113 13065 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:13.648298 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.648711 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 252) took 1344 ms. Trace:
I20260709 13:15:13.621465 13066 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:15:13.650522 11366 rpcz_store.cc:276] 0709 13:15:12.337462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.788469 (+451007us) service_pool.cc:224] Handling call
0709 13:15:12.788854 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.974487 (+185633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.633900 (+659413us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54001e64d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:13.634092 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425633152}
I20260709 13:15:13.652612 11439 rpcz_store.cc:276] 0709 13:15:12.304143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.304296 (+   153us) service_pool.cc:224] Handling call
0709 13:15:12.336277 (+ 31981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:13.048829 (+712552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.648443 (+599614us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b5400315bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:13.648619 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1458827136}
I20260709 13:15:13.656296 13108 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:13.656735 13108 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:13.655989 11438 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:13.661186 13078 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:13.654918 13138 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:15:13.662568 13138 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:13.662894 13138 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:13.664467 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.670573 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:13.666427 13098 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.671635 13041 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.657552 13108 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.677115 13090 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:13.677541 13090 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:13.678473 13090 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.681972 13108 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.682720 13090 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.687175 13055 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.695601 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:13.685688 13087 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.695529 13056 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:13.698120 13056 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:13.698191 13087 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:13.699294 13079 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:13.699358 13056 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.700356 13065 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:13.700657 13056 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.702329 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:13.663726 13138 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.706827 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:13.709405 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:13.710132 11517 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:13.712680 12032 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:13.717505 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:13.717867 11268 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56588 (request call id 234) took 2371 ms. Trace:
I20260709 13:15:13.718235 11268 rpcz_store.cc:276] 0709 13:15:11.346899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.347045 (+   146us) service_pool.cc:224] Handling call
0709 13:15:13.717782 (+2370737us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181698,"reported_tablets":2}
I20260709 13:15:13.720564 11363 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:13.709617 11439 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.722648 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.721072 13099 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:13.726063 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.733152 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.733892 11441 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:13.731750 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.749615 11437 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:13.722626 13127 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:13.724105 13099 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:13.759546 13040 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:13.751266 13081 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:13.758276 13076 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:13.773523 13138 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.770046 13127 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.793035 13085 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:13.800433 13040 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.800981 13040 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.802983 13040 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:13.803464 13098 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.805586 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:13.807612 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.809515 13129 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.810052 13129 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.812111 13129 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:13.813637 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:13.814980 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.788383 13081 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:13.841112 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 112181us
W20260709 13:15:13.843506 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 56587.1us
I20260709 13:15:13.849021 11364 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.795076 13127 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:13.799005 13085 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:13.851866 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:13.865047 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.865485 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.865963 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.866894 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:13.867699 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.868284 11437 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:13.872079 13084 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:13.901484 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.938669 13138 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:13.939360 13131 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.854887 13089 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:15:13.941215 12034 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:13.943146 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.876693 13069 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:13.947065 11368 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:13.950153 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.950569 11366 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:13.950007 13069 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.954073 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:13.939253 13138 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:13.956770 13069 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:13.958244 13138 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:13.936544 13160 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:13.962342 11362 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:13.963732 11444 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:13.943605 13084 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:13.966746 11335 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:13.967084 13084 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:13.953624 13050 consensus_meta.cc:287] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.626s	user 0.003s	sys 0.008s
I20260709 13:15:13.972386 13080 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:13.975642 13073 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:13.976694 13080 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.977095 13080 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.973268 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:13.974135 13138 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:13.981029 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:13.981170 11442 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:13.947796 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:13.987761 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:13.990263 11438 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:13.991529 13038 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:13.990094 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:13.974273 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 94142.9us
W20260709 13:15:14.000732 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 121599us
I20260709 13:15:14.002405 13138 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:14.003621 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.004221 12068 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.004647 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:13.984203 13159 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:15:14.005381 11366 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:14.008344 13162 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:14.010702 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.009829 13162 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:14.012738 13162 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.013707 13131 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.014391 13127 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.016196 13162 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.016917 13076 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:14.018067 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.018016 13162 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.021021 11368 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:14.021219 13162 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.021337 11513 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.018970 11365 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.023617 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:14.024196 11513 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.025481 11515 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:14.025400 13035 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:14.017555 13127 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.025604 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.028470 11439 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.026506 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.031006 13141 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.032266 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:14.033732 13159 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:14.009112 13081 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.034849 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:14.035117 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.037182 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.037833 11365 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.038069 13163 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.038589 13163 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.038902 13163 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.040684 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.040987 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.043349 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:14.044250 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:14.049253 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:14.038807 12986 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.043119 13091 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.052150 13072 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.046083 13162 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:14.059861 13072 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:14.061160 13072 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.061702 13132 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:14.062110 13132 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:14.061116 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.052318 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.063196 11366 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.065923 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.065845 13080 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.070369 11439 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:14.070924 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:14.071544 12060 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:14.072844 11444 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:14.073346 13115 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.060158 13091 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.075340 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.076645 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.077389 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.079610 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:14.076928 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:14.081091 13105 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:14.081635 13105 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:14.084928 11443 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.085860 11444 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.072911 13141 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.079550 13091 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.087227 13127 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:14.065395 13132 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.075906 13080 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.091777 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.090838 13042 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.095731 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:14.098804 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.099728 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.100253 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:14.100977 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.103299 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:14.086977 11438 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.103585 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.105367 13123 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:14.106438 13051 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:14.106974 12063 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:14.104816 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:14.108084 12064 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:14.108930 11362 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:14.110726 11361 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:14.114461 12068 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:14.137545 13058 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:14.114564 11369 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:14.142329 12065 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:14.116180 11370 raft_consensus.cc:2393] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.087141 13087 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.092643 13127 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:14.082446 13105 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.122671 13111 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:14.161610 13111 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.149053 13087 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:14.163017 13087 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.163641 13111 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.164742 13075 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.165220 13075 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.167128 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.167723 11436 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:14.168116 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.168465 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.169152 11364 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:14.171540 11335 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:14.172148 12969 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.172319 13079 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.172724 13079 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.173034 13079 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.173434 13085 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.174405 11515 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:14.177958 13075 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.178351 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:14.125049 13132 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.125604 13033 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:14.126194 13071 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:14.126663 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.129056 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.183007 13071 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:14.184037 12033 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:14.185170 13071 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.186005 11518 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:15:14.112874 13163 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.186909 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.187297 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.175146 13085 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.192507 11515 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:14.193707 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:14.194698 13094 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.195081 13094 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.199318 13094 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.201735 11367 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.129233 11366 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:14.131505 11444 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:14.205411 12033 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.087734 11513 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:14.133810 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.210093 11335 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:14.121236 11484 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:14.149902 12953 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:14.155081 13099 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:14.157351 11363 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1402) took 1947 ms (client timeout 1695 ms). Trace:
I20260709 13:15:14.214076 11440 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:14.157534 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 256) took 1096 ms. Trace:
I20260709 13:15:14.151777 13127 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.214689 11442 rpcz_store.cc:276] 0709 13:15:13.060749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.060844 (+    95us) service_pool.cc:224] Handling call
0709 13:15:13.061077 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:13.981586 (+920509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.157308 (+175722us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54003e4ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:14.157478 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115210496}
I20260709 13:15:14.128245 13122 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.181465 13033 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.207899 13138 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:14.137615 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:14.220775 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 83202.4us
I20260709 13:15:14.221762 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.211079 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:15:14.217921 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 96790.5us
I20260709 13:15:14.216367 13071 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:14.214344 11363 rpcz_store.cc:269] 0709 13:15:12.209465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.401070 (+191605us) service_pool.cc:224] Handling call
0709 13:15:14.157186 (+1756116us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003228d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:14.157277 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2808440064}
I20260709 13:15:14.218416 12969 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.153378 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.224094 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:14.218608 13091 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.222772 13050 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:14.241561 12969 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:14.229015 13083 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.227221 13056 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.254483 13056 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:14.255488 13056 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.257972 13072 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:14.220831 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.218878 13038 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:15:14.261583 12062 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:14.226902 13138 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:14.263721 13138 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.249342 13055 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:15:14.266938 13129 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.267131 12969 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.267443 13129 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.269385 13129 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.267648 13033 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.271270 11334 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:14.271667 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.272285 11442 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:14.272615 12953 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.273190 12953 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:14.274142 12953 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.231187 13071 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:14.280306 13071 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.282783 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.284544 11515 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:14.238381 13051 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.246658 13101 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.288239 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.289661 11442 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:14.290388 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.291145 11515 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 2: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:14.286475 13129 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:14.227363 13058 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:15:14.250108 13042 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.289047 11439 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:14.252162 13083 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:14.238786 13057 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:15:14.255378 11438 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:14.303550 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.246611 13061 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:15:14.313040 11363 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.313392 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:14.322620 13084 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.323290 13084 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.324548 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.325551 13084 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.327207 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.327803 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.328481 11367 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.328402 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.329124 12064 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.330411 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:14.331528 13041 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.331919 13041 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.332301 13041 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.328913 13084 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.342314 13084 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.298615 13129 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.374472 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:14.300091 13101 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:14.306907 13083 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.380383 11484 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:14.380795 12959 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:15:14.303112 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.301486 13141 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:14.392243 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 146038us
I20260709 13:15:14.291960 13051 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.369453 13109 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:14.372116 13131 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.381702 13129 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:14.304715 13042 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.417029 13163 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.416761 13101 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.418677 13094 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.398952 13060 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:15:14.423594 13072 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.424950 13091 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.387306 13055 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:15:14.410462 11518 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:14.430274 13041 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.432123 11363 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.439381 13084 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.431864 13131 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.452256 13111 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.441953 13085 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.402591 13162 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:14.443926 13051 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.387220 13110 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.445840 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.458066 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.458353 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.458082 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.461030 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.507743 12068 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.507853 11369 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:14.467094 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 96590.8us
I20260709 13:15:14.485073 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:14.488476 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:14.490500 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.408661 13109 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:14.464674 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.511626 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.514153 11442 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:14.504637 13111 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.519521 13111 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.524225 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.524408 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.525048 12066 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.490949 13043 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:14.527321 11437 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:14.527325 11369 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.529013 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.534628 12065 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.525763 11366 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.537227 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:14.539194 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.521209 13109 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:14.553887 12986 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:14.554069 13083 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.554499 12986 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.556480 13094 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.557744 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.535274 13161 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.558568 11366 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.547353 13062 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.561514 13062 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:14.529887 13091 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.531167 13162 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.562479 13062 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:14.535418 13056 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.543709 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.547480 13069 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.585741 13056 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.585834 11518 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:14.579437 11263 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39444 (request call id 349) took 1544 ms. Trace:
I20260709 13:15:14.506496 13085 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.587869 13056 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.589710 13085 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.591180 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.591913 11440 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:14.590201 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:14.592880 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.593567 12067 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:14.595265 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.543322 13111 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.599531 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.601080 11366 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:14.530721 12063 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:14.558475 13161 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.622350 13081 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.627676 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.537451 13043 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.580892 13162 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.602274 13050 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:14.638185 13050 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.638633 13050 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.600910 13111 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.640201 13111 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.602021 13069 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:14.641465 13069 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.644758 13083 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.616030 13127 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:14.625953 13056 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.631600 13091 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:14.655823 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.660892 11437 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:14.659019 11408 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:14.637423 13162 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.587167 11263 rpcz_store.cc:276] 0709 13:15:13.034996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.035145 (+   149us) service_pool.cc:224] Handling call
0709 13:15:14.579353 (+1544208us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:13.967764 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:13.968397 (+   633us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:13.968443 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:15:13.968461 (+    18us) tablet.cc:662] Decoding operations
0709 13:15:13.968837 (+   376us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:13.968893 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:13.968925 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:13.969117 (+   192us) tablet.cc:701] Row locks acquired
0709 13:15:13.969135 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:15:13.969282 (+   147us) write_op.cc:270] Start()
0709 13:15:13.969393 (+   111us) write_op.cc:276] Timestamp: P: 1783602913969231 usec, L: 0
0709 13:15:13.969417 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:15:13.970186 (+   769us) log.cc:844] Serialized 530 byte log entry
0709 13:15:14.295731 (+325545us) op_driver.cc:464] REPLICATION: finished
0709 13:15:14.498249 (+202518us) write_op.cc:301] APPLY: starting
0709 13:15:14.498331 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:14.501889 (+  3558us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:14.514286 (+ 12397us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:14.514526 (+   240us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:14.514686 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:15:14.514725 (+    39us) write_op.cc:312] APPLY: finished
0709 13:15:14.514912 (+   187us) log.cc:844] Serialized 23 byte log entry
0709 13:15:14.515088 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:14.515166 (+    78us) write_op.cc:454] Released schema lock
0709 13:15:14.515202 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:14.543180 (+ 27978us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:14.546789 (+  3609us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":475322,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":199235,"apply.run_cpu_time_us":1953,"apply.run_wall_time_us":30429,"num_ops":1,"prepare.queue_time_us":228267,"prepare.run_cpu_time_us":5734,"prepare.run_wall_time_us":48041,"raft.queue_time_us":142477,"raft.run_cpu_time_us":24204,"raft.run_wall_time_us":202203,"replication_time_us":283332,"thread_start_us":450360,"threads_started":4,"wal-append.queue_time_us":87985}]]}
I20260709 13:15:14.662026 13056 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.680286 13041 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.697558 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:14.698231 11515 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.637202 13081 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.648546 13127 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.689651 13061 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:14.715644 13042 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.707903 13094 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.716302 13094 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.717715 13161 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.664886 13087 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.718284 13094 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.699327 13058 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.718936 13058 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.720631 13041 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.721104 13041 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:14.722214 13079 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.005s
I20260709 13:15:14.722981 13041 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.726315 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.726899 11369 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:14.727032 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.729125 11519 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:14.730489 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.729770 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.731290 11409 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:14.731772 11437 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:14.732816 11515 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:14.729365 11335 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:14.733800 13094 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.734150 13094 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.734472 13094 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.720865 13058 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.734560 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:14.722605 13079 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:14.713765 13061 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.740128 11442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:15:14.739239 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.741055 12067 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:14.742081 13056 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.739571 13075 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:14.742575 13075 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.742952 13075 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:14.744179 13127 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.745440 13079 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.745985 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.746559 11438 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.735195 13098 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:14.747108 13098 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.748993 13087 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.742393 11510 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:14.750546 13081 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.751490 13127 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:14.751986 13127 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.752586 13075 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.758038 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:14.759114 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.760341 11369 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.762892 11442 raft_consensus.cc:606] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:14.763083 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.764782 13043 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.765493 12060 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:14.767316 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.768044 11368 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.768775 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.770845 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.768035 12986 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:14.772570 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.773156 11370 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:14.774261 13050 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.774587 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:14.775135 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.776036 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.776701 11441 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:14.740562 13061 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.777882 11363 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.779346 11442 raft_consensus.cc:614] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:14.732569 13111 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.771449 11366 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:14.781699 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:14.756510 13098 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:14.784428 13111 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.784583 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:14.780735 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.775220 11370 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:14.786629 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:14.791814 11517 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:14.776664 13129 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.793555 12986 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:14.793933 12986 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.794294 12986 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.794935 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.795549 11440 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:14.789835 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.796671 11441 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:14.781853 13110 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:14.799589 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:14.800168 11512 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:14.803081 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.800143 11370 raft_consensus.cc:606] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:15:14.795486 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.805308 11441 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:14.802809 12986 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.818157 12069 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.759208 13127 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.824610 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:14.827214 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:14.759123 13075 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.832180 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:14.834404 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:14.835208 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.836038 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:14.836391 11367 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:14.794090 13129 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.840498 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:14.841434 11367 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.843473 11363 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:15:14.843623 12068 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:14.823668 13159 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:14.776894 13091 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.840955 13041 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.827997 13057 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.828172 13132 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:14.786104 13109 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.830096 12953 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:14.840231 13083 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:14.830875 13043 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:14.826478 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:14.829905 13134 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:14.840395 13110 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.854686 12069 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:14.859529 11367 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:14.828527 13105 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:15:14.853143 13041 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:14.830444 13060 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:15:14.838647 13055 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:15:14.876962 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:14.903355 11511 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:14.901106 12067 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:14.879001 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:14.879133 11440 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:14.847513 13159 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.850797 13050 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:14.883953 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:14.856623 12953 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.893657 12959 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:14.895043 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.915074 11519 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:14.895898 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.916064 11516 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:14.867717 13043 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.909200 11443 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:15:14.916994 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:14.853523 13094 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.918423 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.859200 13083 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:14.919620 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.972318 11519 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:14.973238 11516 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:14.907893 13159 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:14.914439 11439 raft_consensus.cc:2368] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4.
I20260709 13:15:14.976284 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.968948 13192 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.858762 13132 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.856273 13057 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:14.980532 13109 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:14.904923 13051 raft_consensus.cc:2955] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:14.916078 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:14.975867 13079 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.983755 13038 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:14.901304 13110 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:14.985087 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:15.049790 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:14.985998 13163 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:14.990923 11443 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:15.007684 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:15.018540 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:14.983067 11438 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:14.876117 13061 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:14.971145 13062 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:14.919338 12953 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:14.976639 13043 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:15.038017 13109 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.047673 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:15.039696 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:15.056239 11516 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:14.974706 13056 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:15:15.068755 11519 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:14.976302 13083 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:14.985905 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 139627us
I20260709 13:15:15.078498 11437 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:15.055334 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:15.023167 13192 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.085853 11335 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:15.087134 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 169313us
I20260709 13:15:15.050513 13079 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.094825 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:15.075292 12986 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.100147 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:15.084903 11512 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:15.108127 12034 leader_election.cc:400] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4.
I20260709 13:15:15.109489 11516 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:15.110533 11442 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:15.083531 13062 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:15.117940 12034 leader_election.cc:403] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:15.062480 13163 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.113868 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:15.119168 11436 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:15.042856 13041 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.092665 13080 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:15.088050 13105 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:15:15.132411 12062 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:15.132992 13055 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:15.133440 13055 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.133778 13055 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:15.133991 13062 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.117627 13081 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:15.135766 13081 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:15.136219 11436 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
W20260709 13:15:15.116888 13188 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:15:15.136657 13081 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:15.120393 13094 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.139559 12986 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:15.141664 13110 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.098851 13050 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:15.143831 13050 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.144258 11440 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:15.145926 13050 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:15.117095 13186 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:15:15.151158 12986 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4.
W20260709 13:15:15.127702 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 74079.5us
I20260709 13:15:15.080492 13109 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.128357 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:15.129489 13057 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.116865 13058 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:15:15.131083 13079 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.118813 11514 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 2: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:15.155099 11335 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:15.158207 11519 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:15.159204 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 112594us
I20260709 13:15:15.162006 11436 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:15.164973 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:15.119789 13123 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:15.167511 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:15.167536 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:15.174566 11519 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:15.176651 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:15.177361 11370 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:15.163746 13187 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:15:15.156323 13080 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:14.881340 13134 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:15.194706 13189 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:15:15.185232 13123 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:15.195204 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:15.200481 11367 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:15:15.220422 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 190897us
I20260709 13:15:15.236948 11370 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:15.160665 13159 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.240680 13138 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:15.241235 13138 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:15.241637 13138 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:15.184695 13072 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:15:15.248188 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 148100us
I20260709 13:15:15.196494 13075 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:15.251291 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 83891.3us
I20260709 13:15:15.190371 13196 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:15.197727 13195 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:15.231316 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:15.231041 11512 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:15.273763 13057 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.274335 13123 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.275239 13080 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.277657 11366 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:15.250301 13132 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.286942 13163 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.230819 13201 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:15:15.278133 13195 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.277529 13197 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:15.291237 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:15.292829 13079 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:15.294037 13195 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:15.285125 13057 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:15.297698 12953 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.301947 13134 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.282641 13098 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:15.301177 13162 consensus_meta.cc:287] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.004s
I20260709 13:15:15.313501 13131 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.298600 13192 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.300024 13197 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:15.319011 11444 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:15.322785 12953 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.318418 13132 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:15.333067 13132 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:15.335749 13159 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.336714 13197 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.328688 13109 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.328542 12953 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.353044 12953 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.317453 13162 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.363437 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:15.364909 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:15.318387 13043 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.367499 13098 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.368878 13134 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:15.371594 13061 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.372571 13201 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:15.373128 13201 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:15.374003 13201 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.377897 13083 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.380133 11511 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:15.328814 13055 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.385462 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:15.391845 11517 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:15.385349 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1606) took 1261 ms. Trace:
I20260709 13:15:15.396428 12953 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.425746 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:15.428035 11363 raft_consensus.cc:2368] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3.
I20260709 13:15:15.363354 13132 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.418051 13084 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.444773 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:15.445793 11443 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:15.445170 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:15.448388 11436 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:15.450541 11335 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:15.389508 13195 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.453513 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:15.454162 11363 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:15.399799 11510 rpcz_store.cc:276] 0709 13:15:14.123611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.124669 (+  1058us) service_pool.cc:224] Handling call
0709 13:15:14.124976 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.759599 (+634623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.385089 (+625490us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54002a76d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:15.385288 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409390464}
I20260709 13:15:15.459048 13132 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:15.459754 13209 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:15.455137 13057 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.476058 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:15.476771 11368 raft_consensus.cc:2368] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4.
I20260709 13:15:15.473680 11363 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:15.456642 12953 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:15.470198 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:15.541834 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.468773 13083 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:15.450203 13084 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.458519 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:15.481667 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 219404us
I20260709 13:15:15.474824 13115 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.488900 12953 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.469141 13075 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:15.504743 13043 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:15.683933 11363 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:15:15.823959 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.020s	sys 0.047s
W20260709 13:15:15.829339 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.020s	sys 0.047s
W20260709 13:15:15.833488 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 666031us
W20260709 13:15:15.684033 13207 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
W20260709 13:15:15.807524 13101 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.091s	user 0.012s	sys 0.003s
I20260709 13:15:15.590628 13196 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:15.796600 13215 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.593225 13050 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:15.787165 13162 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.614289 13072 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:15.827790 12953 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:15.621780 13085 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:15.855859 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:15.885200 11368 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:15.772046 13055 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:15.505862 13209 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.889094 13038 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:15.889627 13038 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.890178 13209 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:15.613437 13138 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.892372 13138 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:15.833838 13115 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:15.894997 13115 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:15.800796 13214 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:15.899111 13214 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.880200 13208 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.833370 13057 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:15.850917 13101 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.012s	sys 0.003s
I20260709 13:15:15.880247 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:15.876075 13215 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.887601 13085 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:15.835518 12062 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
W20260709 13:15:15.852541 13081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.022s	sys 0.039s
W20260709 13:15:15.906961 13080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.001s	sys 0.028s
I20260709 13:15:15.885617 13195 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:15.902980 13041 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.009s	sys 0.021s
W20260709 13:15:15.913518 13080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.001s	sys 0.029s
I20260709 13:15:15.920076 11362 raft_consensus.cc:2368] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3.
I20260709 13:15:15.921051 13196 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.922163 13038 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:15.915597 13041 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.010s	sys 0.021s
I20260709 13:15:15.884326 13050 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:15.913151 13081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.023s	sys 0.040s
I20260709 13:15:15.906633 13062 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:15.886384 13072 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.909410 13208 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.909436 13057 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:15.942642 13197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.019s	sys 0.017s
I20260709 13:15:15.929687 13218 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:15.960315 13218 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.947188 13201 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:15.946859 13041 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:15.967068 13201 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.968559 13218 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:15.959496 13197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.020s	sys 0.017s
I20260709 13:15:15.914669 13075 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.025508 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.029304 11368 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:16.027302 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.033038 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.030784 11368 raft_consensus.cc:2955] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:16.033807 11362 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:16.036322 13050 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.036907 13050 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.037372 13050 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.038173 13141 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.038861 13141 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:15.943444 13062 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:16.006740 13195 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:16.060560 13219 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:15:16.061996 13224 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:15:16.072078 13072 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.072679 13084 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:16.073570 13085 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.079754 13215 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.080808 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:16.081697 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1441) took 1241 ms. Trace:
I20260709 13:15:16.082063 11366 rpcz_store.cc:276] 0709 13:15:14.840403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.843109 (+  2706us) service_pool.cc:224] Handling call
0709 13:15:14.845467 (+  2358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:15.283029 (+437562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.081459 (+798430us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400249854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:16.081621 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993564928}
I20260709 13:15:16.084123 11515 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:16.089186 13141 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.091022 11515 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:16.083266 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:16.092113 11366 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:16.093106 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.094309 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.095679 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.097311 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1620) took 1073 ms. Trace:
W20260709 13:15:16.144621 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 60005.6us
W20260709 13:15:16.156352 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 75612.1us
I20260709 13:15:16.096228 11366 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:16.097645 11515 rpcz_store.cc:276] 0709 13:15:15.024274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.030840 (+  6566us) service_pool.cc:224] Handling call
0709 13:15:16.083609 (+1052769us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b5800010b98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:16.083770 (+   161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:16.097059 (+ 13289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.097254 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11075968}
I20260709 13:15:16.161669 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:16.188580 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.186846 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:16.007978 13041 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.059288 13038 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.253357 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.274712 11366 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:16.073993 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:16.249171 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:16.281234 12067 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.266444 12062 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 1 LEARNER]: Advancing to term 3
I20260709 13:15:16.189850 13084 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:15:16.264365 13226 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.192s	user 0.003s	sys 0.000s
I20260709 13:15:16.278475 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:16.301193 11443 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:16.285820 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:16.248028 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:16.304080 12068 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:16.305063 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:16.305136 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:16.214892 13115 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:16.160372 13214 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:16.266192 13225 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:15:16.250844 11444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1747) took 1455 ms (client timeout 1618 ms). Trace:
I20260709 13:15:16.218470 13089 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:16.300120 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:16.214828 13141 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:16.310737 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:16.404094 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:16.254261 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 385) took 1359 ms. Trace:
I20260709 13:15:16.248299 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:16.266599 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.266853 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:16.448518 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:16.324033 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:16.190181 13209 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:16.352943 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:16.215168 13138 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:16.215955 13162 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:16.280391 13038 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.363766 11483 leader_election.cc:400] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3.
I20260709 13:15:16.465538 11483 leader_election.cc:403] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:15:16.074739 13195 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:16.360476 11444 rpcz_store.cc:269] 0709 13:15:14.795139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.795248 (+   109us) service_pool.cc:224] Handling call
0709 13:15:16.250616 (+1455368us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400278dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:16.250752 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181609344}
I20260709 13:15:16.390425 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:16.429069 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:16.467937 11441 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:16.406011 11517 rpcz_store.cc:276] 0709 13:15:14.894338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.894479 (+   141us) service_pool.cc:224] Handling call
0709 13:15:16.253884 (+1359405us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54000645d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:16.254135 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967470848}
I20260709 13:15:16.318682 11370 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:16.391101 13089 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.451299 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:16.311507 13109 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:16.430469 11361 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:16.462881 11445 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:16.312160 13193 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.389333 13214 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.387943 13060 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.451036 11440 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:16.471159 11370 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:16.502441 12068 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:16.501910 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1618) took 1527 ms. Trace:
I20260709 13:15:16.360832 13115 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.464514 13138 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:16.538286 11333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.022s	sys 0.047s
W20260709 13:15:16.621280 11333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.022s	sys 0.048s
I20260709 13:15:16.463328 12060 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:16.542627 13208 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.478829 13089 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.472378 13050 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.477715 13109 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.441820 13141 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.465201 11365 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
W20260709 13:15:16.538789 12953 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.003s
W20260709 13:15:16.536357 13230 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.464s	user 0.000s	sys 0.001s
I20260709 13:15:16.524739 13057 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.504498 13170 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:16.604990 11408 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.006s	sys 0.001s
I20260709 13:15:16.507234 11442 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:16.475693 13162 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.537659 13043 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:16.539433 13072 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.471880 13209 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.611752 11511 rpcz_store.cc:276] 0709 13:15:14.974492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.976024 (+  1532us) service_pool.cc:224] Handling call
0709 13:15:16.500973 (+1524949us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54000668d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:16.501846 (+   873us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903813376}
I20260709 13:15:16.628530 13201 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:16.623864 13138 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.599128 13127 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.596627 13084 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:16.624910 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:16.624368 13196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.000s	sys 0.007s
I20260709 13:15:16.623212 13214 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.635088 13062 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:16.708328 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 398853us
I20260709 13:15:16.709576 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:16.710548 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:16.710951 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 261631us
I20260709 13:15:16.389070 13229 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:16.713299 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 360460us
W20260709 13:15:16.673949 13051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.014s	sys 0.018s
I20260709 13:15:16.620267 13193 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.691143 12067 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
W20260709 13:15:16.708843 13196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.000s	sys 0.008s
I20260709 13:15:16.678951 13109 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.689419 13236 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:16.629707 13060 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.711104 11511 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.698083 13041 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:16.713815 13111 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:16.683689 12953 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.723839 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:16.731631 12067 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.693918 13170 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.696472 13043 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:16.719154 13051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.014s	sys 0.019s
I20260709 13:15:16.697780 13072 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.717125 13062 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:16.706591 13127 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.716133 13235 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.718346 13239 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.709287 13084 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.739285 13248 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:16.751125 13056 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.751646 13056 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.753574 13056 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.754472 13062 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:16.738541 13228 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:15:16.737519 13041 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:16.759513 13041 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.739109 13111 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:16.762019 13222 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.931s	user 0.009s	sys 0.011s
I20260709 13:15:16.757928 13235 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:16.766440 13227 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:15:16.767669 13235 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.772279 13209 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:16.772975 13109 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.774333 13127 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.775302 13248 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:16.775861 13162 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:16.775977 13109 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.719530 13229 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.763212 13239 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.783432 13111 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.774462 13193 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.787164 13236 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3.
I20260709 13:15:16.776777 13195 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.787021 13229 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.788829 13229 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:16.782310 13254 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.783190 13214 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.799755 13254 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.802297 13254 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.792272 13239 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:16.631752 11368 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.003s	sys 0.025s
W20260709 13:15:16.830318 11368 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.006s	sys 0.026s
I20260709 13:15:16.839602 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.841651 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.830734 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:16.678539 11408 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.007s	sys 0.001s
W20260709 13:15:16.848146 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 941727us
W20260709 13:15:16.621296 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.000s	sys 0.006s
W20260709 13:15:16.851827 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.001s	sys 0.006s
I20260709 13:15:16.856623 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 284) took 1404 ms. Trace:
I20260709 13:15:16.857108 11436 rpcz_store.cc:276] 0709 13:15:15.451806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.452105 (+   299us) service_pool.cc:224] Handling call
0709 13:15:15.452382 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:15.474503 (+ 22121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:15.474532 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:15.504638 (+ 30106us) log.cc:844] Serialized 26 byte log entry
0709 13:15:15.933851 (+429213us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:15.933891 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.933972 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.856059 (+922087us) raft_consensus.cc:1685] finished
0709 13:15:16.856098 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.856360 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22553,"raft.queue_time_us":52473,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":85807,"thread_start_us":228068,"threads_started":1,"wal-append.queue_time_us":827253}
I20260709 13:15:16.860646 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:16.633052 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:16.671506 11439 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.009s	sys 0.025s
W20260709 13:15:16.870122 11439 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.009s	sys 0.025s
I20260709 13:15:16.871323 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 281) took 1778 ms. Trace:
I20260709 13:15:16.691992 11445 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:16.701879 11441 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:16.877179 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:16.877451 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 151859us
I20260709 13:15:16.878476 11517 raft_consensus.cc:2368] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3.
I20260709 13:15:16.879320 12034 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:16.880201 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 516465us
I20260709 13:15:16.881553 11483 leader_election.cc:400] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4.
I20260709 13:15:16.881872 11483 leader_election.cc:403] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:15:16.883011 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:16.905431 12953 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.885651 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:16.906765 12953 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:16.871770 11439 rpcz_store.cc:276] 0709 13:15:15.092534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.093679 (+  1145us) service_pool.cc:224] Handling call
0709 13:15:15.887994 (+794315us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:15.888197 (+   203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:15.888551 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:15.888608 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.027028 (+138420us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:16.027590 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.870843 (+843253us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:16.870956 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.871049 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.871125 (+    76us) raft_consensus.cc:1685] finished
0709 13:15:16.871139 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.871254 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7335,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":37711488,"thread_start_us":842605,"threads_started":1,"wal-append.queue_time_us":772211}
I20260709 13:15:16.887302 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:16.907292 12953 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:16.907887 11510 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.908828 12953 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.892298 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:16.897375 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.898466 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.914707 13035 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:16.916136 13035 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.916335 13041 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:16.916898 13035 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.917490 13042 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:16.918560 13042 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:16.919267 13042 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.750497 13038 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.912164 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:16.923233 12066 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:16.923645 13035 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.924269 13035 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:16.925197 12066 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:16.925251 13035 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.928290 13035 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.923207 13042 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:16.947479 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:16.931202 12062 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.625s	user 0.012s	sys 0.004s
W20260709 13:15:16.951297 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 50283.9us
I20260709 13:15:16.930605 11263 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:16.960464 11442 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:16.961227 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:16.784080 13132 consensus_meta.cc:287] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 1.279s	user 0.006s	sys 0.005s
I20260709 13:15:16.977902 13239 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:16.978214 13193 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:16.978442 13050 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.979094 13193 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.980024 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:16.977931 13056 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.981809 13193 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.979156 13050 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.982092 13056 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:16.984722 13056 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:16.977936 13240 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:16.985882 12032 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.029s (SENT) [suppressed 5 similar messages]
I20260709 13:15:16.986490 13240 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.987627 13162 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:16.989025 13240 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.982542 13055 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:16.996229 13055 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.996465 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:16.996766 13055 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:16.997318 11517 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:16.998040 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:16.999019 11514 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:16.999229 13062 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.000051 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.000803 11516 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:17.000968 12067 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:16.993614 12063 raft_consensus.cc:2410] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:16.993345 13239 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:16.994166 13084 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:16.994374 13109 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:16.995270 11511 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:15:17.036804 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.024348 11515 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:17.002578 12066 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:17.027725 11482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.013s	sys 0.008s
I20260709 13:15:16.996881 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:17.044878 11482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.014s	sys 0.009s
I20260709 13:15:17.004711 12062 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:16.785181 13170 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.024591 13162 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:17.034633 13109 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:17.051589 11263 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:17.040436 13041 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4.
I20260709 13:15:17.044339 13042 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.061887 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.062819 11516 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 1.
I20260709 13:15:17.059173 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.066900 11518 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:17.068107 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.068591 12065 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:15:17.068969 11518 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:17.070086 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.002779 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.061088 11367 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:17.064025 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.028450 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 529) took 1587 ms. Trace:
I20260709 13:15:17.059952 11514 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:16.980367 13236 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:15:17.070375 12065 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:15:17.060035 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 199739us
I20260709 13:15:17.001868 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.061815 13042 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.051617 13094 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:17.076200 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.076792 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.081899 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 1942) took 1033 ms. Trace:
I20260709 13:15:17.089563 13109 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.091023 11516 raft_consensus.cc:2376] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4. Re-sending same reply.
I20260709 13:15:17.066399 13115 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.092077 11363 rpcz_store.cc:276] 0709 13:15:16.048352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.067547 (+ 19195us) service_pool.cc:224] Handling call
0709 13:15:16.996561 (+929014us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:17.081836 (+ 85275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29790976}
I20260709 13:15:17.099653 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.079759 12061 rpcz_store.cc:276] 0709 13:15:15.442956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.448083 (+  5127us) service_pool.cc:224] Handling call
0709 13:15:17.007057 (+1558974us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:17.007317 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.007580 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.007610 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:17.007674 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:17.028108 (+ 20434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.028189 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.028290 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2326272}
I20260709 13:15:17.103722 12061 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:17.096964 11409 leader_election.cc:400] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3.
I20260709 13:15:17.098104 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.104928 11409 leader_election.cc:403] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:17.105201 13042 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.106372 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.107645 11263 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:17.109622 11263 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:17.089605 13094 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.111349 13261 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:17.112340 13261 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.130365 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.131560 13033 raft_consensus.cc:993] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:17.133777 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.134749 11442 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:17.136081 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.136937 11439 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 1.
I20260709 13:15:17.138631 11441 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:17.139937 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.140625 13050 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.143723 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.144537 11445 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 1.
I20260709 13:15:17.146242 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.147126 11436 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 1.
I20260709 13:15:17.149703 13033 raft_consensus.cc:1081] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:15:17.150144 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:17.151696 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 58482.9us
I20260709 13:15:17.152206 13194 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3.
I20260709 13:15:17.152529 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:17.006520 12060 raft_consensus.cc:2955] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:17.155978 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:17.162892 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.164208 13264 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.164749 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.165071 13264 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.167455 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.138661 13072 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.170940 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:17.172112 13205 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.174223 11442 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.160346 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:17.155249 11263 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:17.143735 13043 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.182266 13050 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:17.182781 13050 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.182437 13238 consensus_queue.cc:1059] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:15:17.183678 13089 consensus_meta.cc:287] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.003s
I20260709 13:15:17.183753 13238 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:17.184646 13109 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.182466 13256 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:17.103605 13162 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:17.186444 11437 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 274) took 2411 ms (client timeout 1820 ms). Trace:
I20260709 13:15:17.169358 13094 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.174665 13115 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:17.184405 13089 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.186753 13256 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.183193 13050 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.184868 13238 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:17.172879 13205 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.192869 13043 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.192428 13060 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:17.188899 11437 rpcz_store.cc:269] 0709 13:15:14.774155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.774913 (+   758us) service_pool.cc:224] Handling call
0709 13:15:17.184909 (+2409996us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54003e53d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:17.185095 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695773952}
I20260709 13:15:17.197608 11510 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:17.198755 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.199051 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.215698 11516 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:17.216007 13264 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:17.216377 13138 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.006s
I20260709 13:15:17.216786 13138 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.217116 13238 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.218068 12068 raft_consensus.cc:2368] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4.
I20260709 13:15:17.218462 13238 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.189095 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.204115 11436 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:17.206379 13072 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:17.206856 13089 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.222128 13042 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.223941 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:17.225437 13042 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:17.225917 13042 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:17.226344 13042 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.207497 13268 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.226670 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.207988 11263 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:17.219861 11442 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:17.210642 13256 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.228529 12034 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:17.202807 13141 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.232393 11366 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:17.232789 13043 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.234876 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:17.235721 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 75508.2us
I20260709 13:15:17.237665 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.232412 11518 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:17.242906 13042 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.240682 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.243494 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.243784 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.244609 13197 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:17.244879 11513 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:17.245546 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.250757 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.275272 13205 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.251068 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.251538 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.252068 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.252161 13138 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.252962 13115 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.254040 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.258179 13131 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.260066 13043 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.251578 13266 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.260890 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.240077 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.263728 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:17.273604 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:17.290084 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:17.227537 13268 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.281030 11515 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.281669 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.257428 13239 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:17.285329 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.293678 11518 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:17.296806 11517 raft_consensus.cc:2376] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3. Re-sending same reply.
I20260709 13:15:17.291994 11335 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:17.283682 13131 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:17.302222 11364 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:17.298954 11510 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:17.277437 13141 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:17.325205 13043 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:17.325364 13264 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:17.310976 13131 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.312933 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 530) took 1869 ms. Trace:
I20260709 13:15:17.332988 12063 rpcz_store.cc:276] 0709 13:15:15.443345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.447945 (+  4600us) service_pool.cc:224] Handling call
0709 13:15:17.310192 (+1862247us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540039fdd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:17.312838 (+  2646us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3222108544}
I20260709 13:15:17.334563 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:17.314702 13262 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:17.335521 11361 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.335650 13262 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:17.336570 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 838) took 1035 ms. Trace:
I20260709 13:15:17.315163 13138 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.316435 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.316776 13268 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.307024 13266 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.291093 13205 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.318631 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 841) took 1016 ms. Trace:
I20260709 13:15:17.293664 13229 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.321020 12062 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:17.322582 13072 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:17.323760 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 538) took 1069 ms. Trace:
I20260709 13:15:17.332777 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 1943) took 1284 ms. Trace:
W20260709 13:15:17.339740 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 55069.5us
I20260709 13:15:17.299316 13197 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:17.338789 12066 rpcz_store.cc:276] 0709 13:15:16.301301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.929302 (+628001us) service_pool.cc:224] Handling call
0709 13:15:17.002327 (+ 73025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.039065 (+ 36738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.336326 (+297261us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b5400611cd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:17.336479 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166464384}
I20260709 13:15:17.340842 12068 rpcz_store.cc:276] 0709 13:15:16.301968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.709311 (+407343us) service_pool.cc:224] Handling call
0709 13:15:17.318406 (+609095us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54004b7154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:17.318561 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167375744}
W20260709 13:15:17.355345 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 65344.4us
I20260709 13:15:17.306730 13115 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.360896 13055 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.361361 13266 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.354482 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:17.306579 13042 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.370626 11437 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:17.358354 13197 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:17.355240 12064 rpcz_store.cc:276] 0709 13:15:16.254601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.254771 (+   170us) service_pool.cc:224] Handling call
0709 13:15:16.255129 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:16.717941 (+462812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.720737 (+  2796us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:16.725644 (+  4907us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:16.727767 (+  2123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.323233 (+595466us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54004e6e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:17.323301 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.323582 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048634112}
I20260709 13:15:17.357635 11366 rpcz_store.cc:276] 0709 13:15:16.048685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.299134 (+250449us) service_pool.cc:224] Handling call
0709 13:15:16.299515 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.252523 (+953008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.332453 (+ 79930us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b5400259754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:17.332694 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083639168}
I20260709 13:15:17.371902 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 416) took 1068 ms. Trace:
I20260709 13:15:17.372269 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 842) took 1070 ms. Trace:
I20260709 13:15:17.372701 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.374955 13050 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.350384 13229 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:17.376636 13256 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:17.338562 13261 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.378809 13055 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.378978 13079 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:17.379184 13109 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:17.382157 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:17.362895 13205 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.339926 13138 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:17.397408 13197 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.383392 13072 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:17.389401 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.391878 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.363016 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:17.397897 11517 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:17.336856 13262 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.407902 11515 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:17.397843 12069 rpcz_store.cc:276] 0709 13:15:16.303314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.028459 (+725145us) service_pool.cc:224] Handling call
0709 13:15:17.051798 (+ 23339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:17.052446 (+   648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.052472 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:17.053800 (+  1328us) log.cc:844] Serialized 548 byte log entry
0709 13:15:17.210582 (+156782us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:17.210615 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.210675 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:17.371555 (+160880us) raft_consensus.cc:1685] finished
0709 13:15:17.371572 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.371711 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":156344,"threads_started":1,"wal-append.queue_time_us":228891}
W20260709 13:15:17.412305 11369 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 373) took 1330 ms (client timeout 1525 ms). Trace:
I20260709 13:15:17.415323 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.423305 11370 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:15:17.425940 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.432008 11444 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:17.432922 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.433594 11510 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:15:17.434904 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 78092.8us
I20260709 13:15:17.435752 13050 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.414029 13229 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.441195 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.441489 11368 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:17.441691 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.425638 13138 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.444085 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.444725 11366 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
W20260709 13:15:17.445834 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 82992.3us
I20260709 13:15:17.445930 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.420287 13115 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.446977 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.447643 13115 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.448557 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2031) took 1132 ms. Trace:
I20260709 13:15:17.449549 13115 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.450253 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.450554 13205 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.448896 11443 rpcz_store.cc:276] 0709 13:15:16.316248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.323726 (+  7478us) service_pool.cc:224] Handling call
0709 13:15:16.323984 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:16.884017 (+560033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.884079 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:16.884148 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:16.884172 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.448194 (+564022us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540027a1d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:17.448236 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.448409 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286628736}
I20260709 13:15:17.451124 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:17.451694 11438 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:17.430119 11369 rpcz_store.cc:269] 0709 13:15:16.066907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.071573 (+  4666us) service_pool.cc:224] Handling call
0709 13:15:17.316132 (+1244559us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:17.397597 (+ 81465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5054208}
I20260709 13:15:17.456827 11510 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:17.451359 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 367) took 2089 ms. Trace:
W20260709 13:15:17.458633 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 76551.4us
W20260709 13:15:17.459985 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45191.2us
I20260709 13:15:17.458155 11367 rpcz_store.cc:276] 0709 13:15:15.361456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.363506 (+  2050us) service_pool.cc:224] Handling call
0709 13:15:15.363755 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.083818 (+1720063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.451127 (+367309us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400166a54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:17.451306 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3902456576}
I20260709 13:15:17.468499 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:17.469493 13162 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:17.471778 13041 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.464759 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.475826 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.472295 13162 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.479583 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.473448 11437 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:17.495926 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.502306 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.503445 12066 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 2 LEARNER]: Advancing to term 3
I20260709 13:15:17.503842 12032 leader_election.cc:400] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3.
I20260709 13:15:17.504168 12032 leader_election.cc:403] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:17.506136 12032 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.506644 12986 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.508745 13258 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:17.513022 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.401902 12062 rpcz_store.cc:276] 0709 13:15:16.302196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.004038 (+701842us) service_pool.cc:224] Handling call
0709 13:15:17.004430 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.350794 (+346364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.369739 (+ 18945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:17.496855 13072 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:17.494527 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.498620 13050 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:17.472323 13041 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.518003 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.541522 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.544584 11333 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.542344 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.551090 12953 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:17.523213 13072 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:17.559480 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 86206.3us
I20260709 13:15:17.539111 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.567869 11483 leader_election.cc:400] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4.
I20260709 13:15:17.568152 11483 leader_election.cc:403] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:17.547564 13089 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.515653 13162 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.519361 13229 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.574513 12066 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:17.558277 13050 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 4 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:17.560956 13072 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.584892 13050 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.569319 11361 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:17.571269 11369 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:17.589354 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.590592 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.591022 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.592298 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.592695 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.593489 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.593799 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.594794 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.595320 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:17.578769 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 71344.6us
I20260709 13:15:17.596776 13110 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4.
I20260709 13:15:17.597530 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:17.598620 13041 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.597812 11518 raft_consensus.cc:2955] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:17.581642 13094 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.599258 11510 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:17.578878 11438 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:17.583226 13109 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:17.599707 13094 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.599701 13196 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.600814 13109 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:17.565441 13115 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:15:17.602694 13115 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:17.603048 13115 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:17.603799 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:17.604010 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.605453 12062 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54370
I20260709 13:15:17.606200 13081 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:17.605501 13256 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.600829 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.607100 13251 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.607574 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:17.608150 13072 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.608644 13109 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.609056 13109 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.609432 11511 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:17.608063 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:17.611100 13266 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:17.611213 13072 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.612262 13266 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:17.613276 13266 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.615275 13089 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.611923 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.616313 11510 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:17.616842 13217 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.556353 13033 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:15:17.598953 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.583612 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.546396 13131 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.615078 13197 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.659772 11517 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
W20260709 13:15:17.658192 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 59352.5us
I20260709 13:15:17.658512 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.603960 13115 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.741357 13115 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.608197 12063 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
W20260709 13:15:17.657303 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 101063us
I20260709 13:15:17.742048 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:17.748998 13042 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:17.749691 13042 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.750478 13110 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:17.751621 13259 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:17.751572 13264 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:17.752526 13259 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:17.594847 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.752563 13264 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:17.652908 13229 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.753823 13259 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.751735 13042 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.625587 13138 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.754721 13264 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:17.755833 13264 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.757483 11516 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:17.652299 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:17.626390 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:17.731652 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.760689 13138 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:17.609438 13109 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.768065 13131 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.773273 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:17.622004 11440 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:17.681051 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:17.731253 13268 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.606817 13081 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:17.615916 13278 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:15:17.769022 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:17.782510 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:17.656610 13227 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:15:17.616547 13277 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:15:17.790537 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.797438 11363 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:17.700420 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:17.606328 12062 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:17.601810 13094 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:17.803256 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:17.806380 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 125335us
W20260709 13:15:17.807350 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 53804.6us
I20260709 13:15:17.868172 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:17.804528 13196 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:17.655787 11365 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:17.655891 12986 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3.
I20260709 13:15:17.657408 13089 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:17.651846 13141 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:17.852607 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:17.892580 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 102137us
I20260709 13:15:17.897265 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:17.857167 13258 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:17.867019 13273 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:15:17.918747 12069 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:17.866518 12062 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:17.926925 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 123844us
I20260709 13:15:17.812793 13033 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.927795 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:17.930775 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.933789 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:17.706542 13217 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:17.883447 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.883463 13141 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.923866 13236 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.957746 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:17.957926 12032 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:17.961694 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:17.927198 12986 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:17.933348 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.945295 13229 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:17.979007 13229 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:17.979508 13229 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:17.981328 13247 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.982208 13247 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.982558 13247 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:17.982903 13247 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.955592 13141 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.947871 13041 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:17.987721 13041 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:17.988219 13041 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:17.952261 12064 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:17.956959 12063 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:15:17.990514 12063 pending_rounds.cc:85] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 4
I20260709 13:15:17.990824 12063 pending_rounds.cc:107] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:15:17.991106 12063 raft_consensus.cc:2889] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:17.922827 13268 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.002946 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:17.998538 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 301918us
I20260709 13:15:17.991961 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 413) took 1673 ms. Trace:
I20260709 13:15:18.016736 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.017388 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:17.996184 13196 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:18.021346 11442 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:18.034101 11440 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:18.002166 13162 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:18.028599 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 160498us
I20260709 13:15:18.053468 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.065423 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.071918 13109 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:18.074255 13109 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.077521 13256 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:18.078107 13256 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.075735 13229 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.080050 13256 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:18.020954 12060 rpcz_store.cc:276] 0709 13:15:16.302657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.005499 (+702842us) service_pool.cc:224] Handling call
0709 13:15:17.005852 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.006166 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:17.657562 (+651396us) log.cc:844] Serialized 14 byte log entry
0709 13:15:17.934733 (+277171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:17.945054 (+ 10321us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:17.952515 (+  7461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.958487 (+  5972us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.975864 (+ 17377us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40874,"raft.queue_time_us":51139,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":9156,"thread_start_us":224273,"threads_started":1}
W20260709 13:15:18.065387 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.996109 13033 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.034826 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:18.098097 11369 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:15:18.100080 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.100332 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:18.075654 13268 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:18.102118 13296 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:15:18.108057 13218 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:18.108618 13196 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:18.109869 11484 connection.cc:256] RPC call timeout handler was delayed by -1.1344183690000591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
I20260709 13:15:18.110369 13072 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:18.123584 11484 connection.cc:256] RPC call timeout handler was delayed by -1.0912183690002166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 13:15:18.123781 13072 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.106926 13247 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.111333 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:18.114264 13141 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.039536 13197 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.120719 11436 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:18.121511 11367 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:18.126260 11484 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:18.128954 13247 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.130409 13264 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:18.130805 13264 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:18.130757 13141 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.132366 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:18.123323 13196 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:18.140486 13111 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:18.140998 13111 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:18.141377 13111 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:18.144299 13251 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.537s	user 0.004s	sys 0.005s
I20260709 13:15:18.144790 13251 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.145408 12065 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:18.148154 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:18.149842 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:18.151347 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 233557us
I20260709 13:15:18.130263 13072 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:18.152935 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 255762us
I20260709 13:15:18.131363 13264 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:18.121891 13218 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.156714 11367 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:18.097179 13162 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:15:18.162704 11436 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:18.162650 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 204550us
I20260709 13:15:18.093812 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:18.166286 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 72709.8us
I20260709 13:15:18.160661 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.050765 13081 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.169287 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:18.164902 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:18.157091 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:18.195181 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.196462 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:18.168812 13197 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:18.156976 13072 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 13:15:18.207130 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.206339 11515 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:15:18.207535 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56647.8us
I20260709 13:15:18.210633 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.185611 13041 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.213797 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.218900 13302 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:18.206598 13197 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.227303 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.205885 12986 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:18.239010 13294 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:15:18.240830 12065 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:18.213483 13262 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.216957 11365 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:18.253723 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.256414 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.257195 11368 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:18.243978 13197 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:18.266510 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.249814 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.249204 12986 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:18.251818 13229 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.258323 11514 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:18.257736 13258 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:18.263331 13115 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:18.276136 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.277763 13218 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.278728 13033 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:18.279803 11408 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.013s	sys 0.015s
I20260709 13:15:18.293398 13111 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.294765 12986 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:18.298599 11408 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.014s	sys 0.015s
W20260709 13:15:18.327371 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 909412us
I20260709 13:15:18.341953 13240 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:18.342625 13240 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.263119 12062 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.322335 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.322832 13111 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:18.323648 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:18.325271 13072 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:18.326078 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:18.273923 13262 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.352097 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.348528 11510 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:18.354271 13138 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.375042 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:18.377225 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 51323.6us
I20260709 13:15:18.361326 13240 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:18.394207 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 260699us
I20260709 13:15:18.416960 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:18.421789 11444 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:18.424137 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 221765us
I20260709 13:15:18.425323 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:18.426783 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:18.427848 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 144875us
I20260709 13:15:18.387323 13138 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.579702 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:18.546592 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 205546us
I20260709 13:15:18.429889 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1295) took 1351 ms. Trace:
I20260709 13:15:18.447505 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:18.538734 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:18.713934 13251 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:18.804889 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 296761us
I20260709 13:15:18.807627 11441 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 4 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:18.685900 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:18.805434 11511 rpcz_store.cc:276] 0709 13:15:17.081568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.475576 (+394008us) service_pool.cc:224] Handling call
0709 13:15:18.429687 (+954111us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540031ea54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:18.429821 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296384256}
W20260709 13:15:18.807974 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:18.804373 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 379119us
W20260709 13:15:18.578898 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 152438us
W20260709 13:15:18.810196 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.478116 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:18.504968 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.804715 11442 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:18.814965 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.812500 13138 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:18.815980 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 368580us
I20260709 13:15:18.816107 11437 raft_consensus.cc:2410] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:18.817418 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:18.807600 13050 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:18.770833 13310 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 0.252s	user 0.001s	sys 0.001s
I20260709 13:15:18.825380 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:18.423776 13041 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:18.699837 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.830695 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:18.548976 13313 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.623878 13247 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.831712 11436 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:18.520156 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 306) took 1062 ms. Trace:
I20260709 13:15:18.843113 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:18.732620 13262 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.522182 13080 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:18.826251 13094 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:18.769146 13304 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:18.518538 13229 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:18.835721 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:18.836932 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.430332 13196 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.840372 13318 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.822824 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1645) took 1092 ms. Trace:
I20260709 13:15:18.846271 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:18.847203 13328 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:18.838150 13314 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:18.847738 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.663308 12062 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.522456 13072 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.314s
I20260709 13:15:18.852432 13080 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:18.527973 13264 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.855144 11368 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:18.838132 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:18.857654 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.863950 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:18.865239 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:18.869860 13197 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.600s	user 0.002s	sys 0.006s
I20260709 13:15:18.870038 13320 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:18.870321 13197 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:18.870672 13320 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.872268 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:18.872885 11436 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:18.851387 13094 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:18.875649 11442 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:18.851727 11438 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:18.875770 11440 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:18.876153 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:18.899689 13331 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:18.456787 13258 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.833806 13050 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.900552 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:54370 (request call id 549) took 1273 ms. Trace:
I20260709 13:15:18.872834 13320 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:18.849843 13247 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.901603 13094 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:18.915159 13217 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:18.917193 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:18.932305 11445 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:18.929461 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:18.928894 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 90899.8us
I20260709 13:15:18.901584 13326 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:18.932351 11363 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:18.901019 13314 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:18.908427 13259 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.910531 11365 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:18.855053 13313 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:18.945919 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 72978.6us
W20260709 13:15:18.945469 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 127083us
I20260709 13:15:18.972517 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:18.966885 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:18.932796 13072 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:18.856441 13304 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.999454 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:15:18.986523 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 176009us
I20260709 13:15:18.875411 13328 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.912066 12062 rpcz_store.cc:276] 0709 13:15:17.602935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.603254 (+   319us) service_pool.cc:224] Handling call
0709 13:15:18.875029 (+1271775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:18.858613 13229 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:18.932483 13320 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:15:18.955229 11438 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:19.008481 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.027941 11334 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:18.993933 13072 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:19.032999 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 62693.8us
I20260709 13:15:19.008523 13316 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:19.041932 11334 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:18.928683 13331 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:19.047537 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 118164us
I20260709 13:15:19.048270 11408 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:19.011487 13315 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:19.052050 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:19.057060 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:19.054978 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:19.039131 13217 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.043699 13259 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:15:19.064882 13333 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:19.043232 13081 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:19.064684 13313 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.066653 13313 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:19.068440 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.067099 13313 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:19.045540 13256 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:19.065056 13330 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:15:19.069727 13313 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:19.055938 13316 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:19.057758 13141 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:18.993777 13326 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:19.092515 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:19.093438 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 121084us
I20260709 13:15:19.103065 11368 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:19.106173 11515 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:19.095825 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.061908 13315 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:19.045420 13314 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:19.089515 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.045324 13080 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.100049 13050 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.101862 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1647) took 1369 ms. Trace:
I20260709 13:15:19.103533 13335 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:18.844659 11439 rpcz_store.cc:276] 0709 13:15:17.444860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.445924 (+  1064us) service_pool.cc:224] Handling call
0709 13:15:17.446223 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.446479 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:17.877071 (+430592us) log.cc:844] Serialized 14 byte log entry
0709 13:15:18.463833 (+586762us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:18.479694 (+ 15861us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:18.495742 (+ 16048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.501430 (+  5688us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:18.507128 (+  5698us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":604731,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":46391,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":14114,"thread_start_us":807034,"threads_started":3,"wal-append.queue_time_us":468048}
I20260709 13:15:19.097846 13264 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:19.104238 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:19.104542 13081 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:19.076697 13197 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:19.098271 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:19.097097 11518 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.009s	sys 0.013s
I20260709 13:15:19.135317 13316 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.179119 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.089691 13217 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:19.190444 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 126832us
I20260709 13:15:19.216936 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:19.183144 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.106875 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:19.215859 11518 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.010s	sys 0.013s
I20260709 13:15:19.216182 12069 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:19.218936 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.217782 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:19.221058 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 114615us
I20260709 13:15:19.177233 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.144369 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:19.106789 13325 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:15:18.859375 11512 rpcz_store.cc:276] 0709 13:15:17.727428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.968311 (+240883us) service_pool.cc:224] Handling call
0709 13:15:17.978798 (+ 10487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:17.979178 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.979208 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:17.979771 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:15:18.210781 (+231010us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:18.215520 (+  4739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.217926 (+  2406us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:18.518661 (+300735us) raft_consensus.cc:1685] finished
0709 13:15:18.518684 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:18.819814 (+301130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9814,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":222765,"threads_started":1,"wal-append.queue_time_us":371732}
I20260709 13:15:19.233023 12067 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 2: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:19.174311 13322 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.091981 13342 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:19.237401 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1455) took 1504 ms. Trace:
I20260709 13:15:19.242502 11369 rpcz_store.cc:276] 0709 13:15:17.732566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.732729 (+   163us) service_pool.cc:224] Handling call
0709 13:15:17.796488 (+ 63759us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.118777 (+322289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.237091 (+1118314us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54000a73d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:19.237341 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656700800}
I20260709 13:15:19.236058 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1300) took 1631 ms. Trace:
I20260709 13:15:19.213842 13313 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:19.243772 11518 rpcz_store.cc:276] 0709 13:15:17.587234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.597169 (+  9935us) service_pool.cc:224] Handling call
0709 13:15:17.597486 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.597691 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:18.223317 (+625626us) log.cc:844] Serialized 14 byte log entry
0709 13:15:19.218722 (+995405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:19.218781 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:19.218833 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.218896 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.219001 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":269363,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":950720,"threads_started":2,"wal-append.queue_time_us":663552}
I20260709 13:15:19.087477 13256 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.221139 11519 rpcz_store.cc:276] 0709 13:15:17.732279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.230444 (+498165us) service_pool.cc:224] Handling call
0709 13:15:18.230800 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.231020 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:18.231046 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:18.248115 (+ 17069us) log.cc:844] Serialized 26 byte log entry
0709 13:15:18.499663 (+251548us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:18.499703 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.499765 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:19.101651 (+601886us) raft_consensus.cc:1685] finished
0709 13:15:19.101671 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.101796 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":246230,"threads_started":1,"wal-append.queue_time_us":803164}
I20260709 13:15:19.309707 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.179883 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.214836 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:19.309576 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 254651us
I20260709 13:15:19.311733 11368 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:19.309782 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:19.174015 13317 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:15:19.342670 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.345036 12067 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:19.332315 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:19.316370 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.220690 13335 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.353621 11515 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:19.134864 13141 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:19.354200 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 136546us
I20260709 13:15:19.219623 11438 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:19.287225 13259 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:19.242405 13322 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.367547 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.239028 12062 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:19.275836 13050 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:19.360265 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.342415 13110 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:19.374400 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 276187us
I20260709 13:15:19.379619 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.378417 12067 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:19.228137 13081 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.353463 13162 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.166174 13072 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:19.359381 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.390846 12068 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
W20260709 13:15:19.384202 13311 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.862s	user 0.003s	sys 0.001s
I20260709 13:15:19.392963 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:19.393376 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.367017 13335 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:19.373004 12061 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:19.386950 12060 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 2: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:19.405081 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.377833 13348 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:19.422564 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:19.431596 12067 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:19.434338 13340 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:15:19.434619 13332 consensus_queue.cc:1059] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:15:19.425287 12069 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:19.425627 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:19.428359 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.428540 11369 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:19.402865 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:19.405129 13297 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:19.379609 13259 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.389700 13266 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:15:19.236215 13315 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.401311 13197 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:15:19.409854 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.433156 13264 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:19.469244 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.464218 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.470880 11369 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
W20260709 13:15:19.459370 11445 consensus_meta.cc:287] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.000s
I20260709 13:15:19.432209 13328 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.466598 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:19.426097 12064 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:19.472069 11445 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:19.438393 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:19.403980 13162 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.401067 13050 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.452025 11439 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:19.442646 13329 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:19.484059 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:19.461650 13110 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.462430 13229 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:19.492460 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:19.472934 13340 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:19.454437 13356 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:19.473382 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:19.504089 13309 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.981s	user 0.002s	sys 0.000s
I20260709 13:15:19.504662 12062 raft_consensus.cc:2393] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 2: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:19.492651 13365 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.505759 13365 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.507892 13365 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.507999 13333 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:19.508426 13333 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.491334 13328 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:19.508754 13333 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:19.509229 13328 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.509668 13328 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.511381 13256 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.511765 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1644) took 1784 ms. Trace:
I20260709 13:15:19.438082 13348 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.512224 11513 rpcz_store.cc:276] 0709 13:15:17.727138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.731663 (+  4525us) service_pool.cc:224] Handling call
0709 13:15:17.860903 (+129240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:17.951379 (+ 90476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.955144 (+  3765us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:18.016800 (+ 61656us) log.cc:844] Serialized 26 byte log entry
0709 13:15:18.326152 (+309352us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:18.326202 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.326284 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:19.476785 (+1150501us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:19.511467 (+ 34682us) raft_consensus.cc:1685] finished
0709 13:15:19.511494 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.511696 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3815,"raft.queue_time_us":43162,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":12940,"spinlock_wait_cycles":77598464,"thread_start_us":233822,"threads_started":1,"wal-append.queue_time_us":354247}
I20260709 13:15:19.498106 13329 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.514469 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:19.516232 13322 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.516608 13050 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.516925 13196 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:19.517131 13360 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:15:19.517381 13329 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.517690 13357 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:19.518218 13357 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:19.519068 13094 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.000s
I20260709 13:15:19.519498 13094 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:19.520351 13357 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:19.521520 13363 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:19.495594 13110 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.498895 13344 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:19.522986 13344 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:19.523381 13344 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:19.495554 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1650) took 1721 ms. Trace:
I20260709 13:15:19.524333 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:19.525408 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:19.525899 11511 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:19.482905 13331 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.527529 13345 raft_consensus.cc:3055] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:19.495458 13264 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:19.496376 11519 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:15:19.492843 12065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 546) took 1890 ms (client timeout 1838 ms). Trace:
W20260709 13:15:19.528465 12065 rpcz_store.cc:269] 0709 13:15:17.601987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.603515 (+  1528us) service_pool.cc:224] Handling call
0709 13:15:19.492681 (+1889166us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54005d6e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:19.492777 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244217600}
W20260709 13:15:19.495373 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.499935 13340 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:19.476593 13236 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:19.530328 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.473069 13115 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:19.580711 13132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.056s	sys 0.103s
I20260709 13:15:19.531839 13110 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:19.587976 11363 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.011s	sys 0.012s
I20260709 13:15:19.524082 11517 rpcz_store.cc:276] 0709 13:15:17.774461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.101138 (+326677us) service_pool.cc:224] Handling call
0709 13:15:18.101520 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.102055 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:18.102086 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:18.102579 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:15:18.258233 (+155654us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:18.258271 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.258341 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:19.383495 (+1125154us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:19.490189 (+106694us) raft_consensus.cc:1685] finished
0709 13:15:19.490212 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.495490 (+  5278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1197,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":72871808,"thread_start_us":155290,"threads_started":1,"wal-append.queue_time_us":327127}
I20260709 13:15:19.591298 13340 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:19.493603 13332 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:19.527949 13345 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:19.509584 13081 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:19.595345 11363 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.011s	sys 0.013s
W20260709 13:15:19.595376 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.598912 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:19.663659 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.591804 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:19.675803 11513 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:19.591233 13316 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:19.660032 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 62441.8us
I20260709 13:15:19.540715 13236 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:19.511768 13328 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.663659 11517 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.588838 13115 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.588438 13229 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.707285 13229 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:19.593050 13132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.057s	sys 0.104s
I20260709 13:15:19.671744 13326 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:19.674676 13348 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.517740 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:19.679361 11511 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:15:19.681839 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 92574.8us
I20260709 13:15:19.683096 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:19.705300 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.706454 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.661334 13354 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.592756 13320 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:15:19.599413 13347 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:19.746590 11439 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:19.765064 13229 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:19.781582 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.794118 11511 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:19.800904 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.803931 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:19.805298 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:19.806458 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:19.748802 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.813026 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:19.813652 11440 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:19.814711 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:19.815431 11370 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:19.812212 13344 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.816512 11438 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:19.826923 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1785) took 1008 ms. Trace:
I20260709 13:15:19.826701 13335 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:19.695887 13217 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.827435 11438 rpcz_store.cc:276] 0709 13:15:18.818770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.815810 (+997040us) service_pool.cc:224] Handling call
0709 13:15:19.816156 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.826617 (+ 10461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.826855 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:19.828269 11370 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:19.792544 13340 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:19.835544 11517 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:19.838210 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:19.838886 11517 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:19.816591 13162 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.841958 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:19.732923 13141 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:19.842260 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:19.839740 11370 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.845855 11370 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.846701 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:19.848074 13368 raft_consensus.cc:2955] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:19.852754 11370 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:19.844854 13259 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:19.853811 11366 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:19.857529 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.858201 11370 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:19.862576 12064 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:19.864331 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1318) took 2001 ms. Trace:
I20260709 13:15:19.864791 11510 rpcz_store.cc:276] 0709 13:15:17.862559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.165929 (+303370us) service_pool.cc:224] Handling call
0709 13:15:18.214687 (+ 48758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:18.374284 (+159597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.864071 (+1489787us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54003d8e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:19.864258 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253336832}
I20260709 13:15:19.866567 13094 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:19.867539 13258 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.867853 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1316) took 2082 ms. Trace:
I20260709 13:15:19.868089 13258 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.760636 13354 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.868316 11514 rpcz_store.cc:276] 0709 13:15:17.788681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.108439 (+319758us) service_pool.cc:224] Handling call
0709 13:15:18.108766 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:18.275478 (+166712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.867606 (+1592128us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540014bed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:19.867786 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308618496}
I20260709 13:15:19.856681 13348 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:19.874907 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:19.890668 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2044) took 1878 ms. Trace:
I20260709 13:15:19.867318 11510 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:19.908216 11441 rpcz_store.cc:276] 0709 13:15:18.012491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.012648 (+   157us) service_pool.cc:224] Handling call
0709 13:15:18.033696 (+ 21048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:18.808600 (+774904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.890354 (+1081754us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54002ea754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:19.890576 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1779447552}
I20260709 13:15:19.900492 13354 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.931593 13344 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:19.931447 13345 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.893301 13348 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:19.932670 13345 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.932989 13348 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:19.933895 13236 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:19.896616 13336 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:19.948290 13247 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:19.951556 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 109365us
I20260709 13:15:19.951442 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1777) took 1134 ms. Trace:
I20260709 13:15:19.952716 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:19.953964 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.954823 11363 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:19.953042 11436 rpcz_store.cc:276] 0709 13:15:18.817018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.873546 (+ 56528us) service_pool.cc:224] Handling call
0709 13:15:18.907624 (+ 34078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:18.912508 (+  4884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:18.912535 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:18.912576 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:18.912610 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.912666 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.951239 (+1038573us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b540028a0d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:19.951359 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77570304}
I20260709 13:15:19.982967 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:19.951390 11513 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:19.950147 13115 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:19.968112 11439 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:20.017227 11519 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:20.018543 11519 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:20.012321 11370 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
W20260709 13:15:20.016038 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 63639.8us
I20260709 13:15:20.020293 11436 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:20.021060 12033 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:20.020331 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.025748 11366 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:20.019840 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:20.026885 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:20.027697 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1779) took 1210 ms. Trace:
I20260709 13:15:20.028782 13302 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:20.029175 13258 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:20.029286 11510 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:15:20.029709 11510 pending_rounds.cc:85] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 5
I20260709 13:15:20.029982 11510 pending_rounds.cc:107] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:15:20.030272 11510 raft_consensus.cc:2889] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:20.030850 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.031457 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.031688 12063 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 2: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:20.032088 12067 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:20.032755 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.032913 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.033970 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.034651 12063 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:20.035889 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.037638 12063 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:20.038844 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 317) took 1199 ms. Trace:
I20260709 13:15:20.039963 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.041232 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:20.042007 13370 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 13:15:20.043047 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:20.043855 11518 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:15:20.044162 13141 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:20.045431 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:20.045665 13357 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:20.046120 13357 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:20.046475 13357 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:20.046998 11409 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:20.048743 11408 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:20.049686 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2066) took 1220 ms. Trace:
I20260709 13:15:20.050072 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:20.050071 11436 rpcz_store.cc:276] 0709 13:15:18.830016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.968861 (+1138845us) service_pool.cc:224] Handling call
0709 13:15:20.017133 (+ 48272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:20.049421 (+ 32288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.049633 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:20.028033 11444 rpcz_store.cc:276] 0709 13:15:18.817455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.992169 (+1174714us) service_pool.cc:224] Handling call
0709 13:15:20.027636 (+ 35467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:20.044252 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1321) took 1008 ms. Trace:
I20260709 13:15:20.052289 11518 rpcz_store.cc:276] 0709 13:15:19.036128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.042791 (+1006663us) service_pool.cc:224] Handling call
0709 13:15:20.044197 (+  1406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:20.054165 11443 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:20.054919 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:20.056295 11408 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:20.056630 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:20.048041 13366 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:15:20.029318 13302 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:20.059090 11444 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:20.059702 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2069) took 1021 ms. Trace:
I20260709 13:15:20.059554 13302 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.060096 11444 rpcz_store.cc:276] 0709 13:15:19.038398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.058481 (+1020083us) service_pool.cc:224] Handling call
0709 13:15:20.058780 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.059465 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.059641 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:20.061053 11368 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:20.017269 13344 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:20.026516 13333 consensus_meta.cc:287] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.000s
I20260709 13:15:20.064684 11440 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:20.065176 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 318) took 1162 ms. Trace:
I20260709 13:15:20.039063 13336 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:20.067423 11443 rpcz_store.cc:276] 0709 13:15:18.892933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.048622 (+1155689us) service_pool.cc:224] Handling call
0709 13:15:20.048906 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.054553 (+  5647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.054728 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:20.066133 11445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 311) took 1524 ms (client timeout 1580 ms). Trace:
I20260709 13:15:20.068197 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2068) took 1030 ms. Trace:
W20260709 13:15:20.073899 11445 rpcz_store.cc:269] 0709 13:15:18.537827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.537931 (+   104us) service_pool.cc:224] Handling call
0709 13:15:20.059373 (+1521442us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54003e5dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:20.062465 (+  3092us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242533120}
I20260709 13:15:20.074095 11436 rpcz_store.cc:276] 0709 13:15:19.038152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.050915 (+1012763us) service_pool.cc:224] Handling call
0709 13:15:20.067855 (+ 16940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.067982 (+   127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:20.068000 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.068026 (+    26us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:20.068046 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.068078 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.068153 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:20.077569 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2065) took 1252 ms. Trace:
I20260709 13:15:20.077855 13316 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:20.078346 13365 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:20.078006 11441 rpcz_store.cc:276] 0709 13:15:18.825075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.934397 (+1109322us) service_pool.cc:224] Handling call
0709 13:15:19.934779 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.934983 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:19.935789 (+   806us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.077079 (+141290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.077179 (+   100us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:20.077239 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.077391 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.077513 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14417,"prepare.queue_time_us":261,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":15059,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":109970,"threads_started":1}
I20260709 13:15:20.062225 13240 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:20.080780 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.062207 13141 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:20.063953 13326 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:20.089954 12066 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:20.083087 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.094242 11368 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:20.064381 13050 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:20.087101 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:20.073635 13336 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:20.102700 12033 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:20.091154 13326 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:20.048936 11439 rpcz_store.cc:276] 0709 13:15:18.839865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.038017 (+1198152us) service_pool.cc:224] Handling call
0709 13:15:20.038328 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.038520 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:20.038545 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.038590 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:20.038626 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.038683 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.038781 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:20.102706 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:20.072394 13333 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:20.107354 13301 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:20.108590 13259 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:15:20.159399 13259 consensus_queue.cc:1243] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:20.159911 13259 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:20.160225 13259 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:20.161414 12034 connection.cc:256] RPC call timeout handler was delayed by -1.0059572360000857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 13:15:20.142953 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:20.144106 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:20.163542 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:20.163879 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.164088 13304 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:20.164639 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:20.165426 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.166109 11510 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:20.145287 13357 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.151777 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.167148 13315 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:20.134042 13115 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:20.167689 13315 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:20.169046 13372 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.425s	user 0.000s	sys 0.001s
I20260709 13:15:20.169251 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.170076 11512 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:20.170416 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1780) took 1352 ms. Trace:
I20260709 13:15:20.175069 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.177202 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.177424 13072 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:20.177893 11510 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:20.179315 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.116808 13335 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:20.161027 13259 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.117136 13326 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:20.187490 11513 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:20.188750 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:20.189090 13312 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.572s	user 0.003s	sys 0.006s
I20260709 13:15:20.189644 11513 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:20.190562 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.191166 11513 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:20.124990 13301 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.168556 13315 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.179953 13236 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:20.205878 11436 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:20.158805 11517 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:20.158274 13333 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:20.179237 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.181823 11519 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:20.229730 11518 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:20.183267 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:20.195147 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 138598us
I20260709 13:15:20.195560 12953 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.197137 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:20.170764 11442 rpcz_store.cc:276] 0709 13:15:18.817678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.908173 (+ 90495us) service_pool.cc:224] Handling call
0709 13:15:18.908588 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.181534 (+272946us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:19.590477 (+408943us) log.cc:844] Serialized 14 byte log entry
0709 13:15:19.778956 (+188479us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:19.779013 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:19.779071 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.169436 (+390365us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b1800124300. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419
0709 13:15:20.170196 (+   760us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54002094d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:20.170249 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.170361 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37532,"prepare.queue_time_us":293258,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":37805,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5704,"spinlock_wait_cycles":658115712,"thread_start_us":491459,"threads_started":2,"wal-append.queue_time_us":229113}
I20260709 13:15:20.183018 12032 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:20.180318 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.167094 13141 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:20.226527 11510 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:20.230706 12953 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.226819 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:20.205929 13369 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:20.191231 13247 consensus_queue.cc:1175] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34a42aa6aa0a4778a532c6433d70b981" responder_term: 4 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:20.168514 13357 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:20.228353 11266 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:20.239053 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:20.234586 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:20.205384 13301 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:20.191282 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1784) took 1372 ms. Trace:
I20260709 13:15:20.247439 11263 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39444 (request call id 350) took 5520 ms. Trace:
I20260709 13:15:20.246783 11440 rpcz_store.cc:276] 0709 13:15:18.818538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.829901 (+1011363us) service_pool.cc:224] Handling call
0709 13:15:20.062532 (+232631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.071675 (+  9143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.191105 (+119430us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54003e5dd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:20.191224 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503918464}
I20260709 13:15:20.234241 13335 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:20.232232 13132 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:20.248526 13316 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:20.238413 13050 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:15:20.244781 13247 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.241353 11512 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:20.267599 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:20.309921 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1791) took 1170 ms. Trace:
W20260709 13:15:20.321420 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 138484us
I20260709 13:15:20.322273 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:20.322852 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.258577 13335 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.260483 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:20.327725 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 81023.2us
I20260709 13:15:20.323279 12032 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:20.267462 13138 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:20.343801 12953 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:20.347455 13343 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.880s	user 0.000s	sys 0.011s
I20260709 13:15:20.343320 11515 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:20.348724 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1658) took 1111 ms. Trace:
I20260709 13:15:20.366210 11517 rpcz_store.cc:276] 0709 13:15:19.237492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.854289 (+616797us) service_pool.cc:224] Handling call
0709 13:15:19.952144 (+ 97855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.206712 (+254568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.348468 (+141756us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400064fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:20.348664 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474793856}
I20260709 13:15:20.349709 12062 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:20.350507 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 1969) took 1527 ms. Trace:
I20260709 13:15:20.351655 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:20.291795 13258 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.333707 13302 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:20.370713 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47487.7us
W20260709 13:15:20.329772 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 273247us
I20260709 13:15:20.373330 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:20.385447 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 202247us
I20260709 13:15:20.370891 12953 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:20.386157 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:20.353381 11513 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:20.354641 13335 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:20.354394 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:20.386261 12953 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:20.260766 13132 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.391355 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:20.391729 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 210103us
I20260709 13:15:20.247893 11263 rpcz_store.cc:276] 0709 13:15:14.722632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.726452 (+  3820us) service_pool.cc:224] Handling call
0709 13:15:20.243149 (+5516697us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:17.812055 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:17.991548 (+179493us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:17.991600 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:15:17.991627 (+    27us) tablet.cc:662] Decoding operations
0709 13:15:18.071210 (+ 79583us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:18.071295 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:18.071324 (+    29us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:15:18.071973 (+   649us) tablet.cc:701] Row locks acquired
0709 13:15:18.072015 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:15:18.072178 (+   163us) write_op.cc:270] Start()
0709 13:15:18.072312 (+   134us) write_op.cc:276] Timestamp: P: 1783602918072119 usec, L: 0
0709 13:15:18.072343 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:15:18.073027 (+   684us) log.cc:844] Serialized 2426 byte log entry
0709 13:15:19.179842 (+1106815us) op_driver.cc:464] REPLICATION: finished
0709 13:15:19.462018 (+282176us) write_op.cc:301] APPLY: starting
0709 13:15:19.473351 (+ 11333us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:19.531534 (+ 58183us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:19.531662 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:19.760881 (+229219us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:19.761231 (+   350us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:15:19.761296 (+    65us) write_op.cc:312] APPLY: finished
0709 13:15:19.761605 (+   309us) log.cc:844] Serialized 53 byte log entry
0709 13:15:19.761800 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.761905 (+   105us) write_op.cc:454] Released schema lock
0709 13:15:19.761972 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:19.762385 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.762411 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":255455,"apply.run_cpu_time_us":4078,"apply.run_wall_time_us":323771,"num_ops":6,"prepare.queue_time_us":446388,"prepare.run_cpu_time_us":20062,"prepare.run_wall_time_us":528437,"raft.queue_time_us":266604,"raft.run_cpu_time_us":37503,"raft.run_wall_time_us":202968,"replication_time_us":1107349,"thread_start_us":654055,"threads_started":4,"wal-append.queue_time_us":318241}]]}
I20260709 13:15:20.316128 11443 rpcz_store.cc:276] 0709 13:15:19.139115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.072326 (+933211us) service_pool.cc:224] Handling call
0709 13:15:20.072617 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.072811 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:20.088305 (+ 15494us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.309550 (+221245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.309605 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:20.309656 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.309724 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.309852 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15723,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":1571,"raft.queue_time_us":5254,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":1013,"thread_start_us":220906,"threads_started":1}
I20260709 13:15:20.291867 13247 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.380894 13229 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:20.416124 11441 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:20.417565 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.418882 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.419574 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.417271 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:20.422663 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.381145 13302 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:20.422710 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:20.424677 12953 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:20.431239 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.390797 13258 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:20.464617 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.428867 13302 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:20.368392 11365 rpcz_store.cc:276] 0709 13:15:18.827015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.828114 (+  1099us) service_pool.cc:224] Handling call
0709 13:15:18.838561 (+ 10447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.971734 (+133173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.350228 (+1378494us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b5400165154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:20.350432 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83047168}
I20260709 13:15:20.413415 13366 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:15:20.497169 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:20.427556 13368 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:15:20.499930 12953 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:15:20.487891 13348 consensus_meta.cc:287] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.000s
W20260709 13:15:20.499040 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 112962us
I20260709 13:15:20.431954 13384 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:15:20.518201 12034 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:20.506294 13318 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:20.506410 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:20.533143 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 101225us
I20260709 13:15:20.499867 13315 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:20.516937 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 125658us
I20260709 13:15:20.519131 13348 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.580607 13357 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.584722 11517 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:20.515374 13050 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:20.582706 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 152144us
W20260709 13:15:20.575649 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 51538.7us
I20260709 13:15:20.508266 13393 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:20.532666 13318 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:20.599174 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 79109.1us
W20260709 13:15:20.598861 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.597999 12953 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:20.539250 13138 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:20.588449 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:20.635857 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47518.5us
I20260709 13:15:20.636579 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:20.521273 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:20.583925 13391 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:15:20.580794 13366 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:15:20.662827 11437 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:20.682919 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 46412.5us
I20260709 13:15:20.621692 13357 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:20.601022 13397 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 13:15:20.629020 13393 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:20.694674 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:20.696718 13141 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:20.710433 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.711311 11370 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:20.696960 13408 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:20.608428 13258 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.663415 13318 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:20.662376 13162 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.705701 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:20.662181 13138 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:20.706561 13387 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:15:20.680536 11513 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
W20260709 13:15:20.706727 13380 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 13:15:20.712807 13382 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:15:20.727710 13162 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:20.698037 13050 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
W20260709 13:15:20.734730 13385 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.265s	user 0.002s	sys 0.002s
W20260709 13:15:20.716852 13389 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260709 13:15:20.717064 13390 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:15:20.739344 11441 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:15:20.739596 13072 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.562s	user 0.004s	sys 0.008s
W20260709 13:15:20.764948 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 232814us
I20260709 13:15:20.740183 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2061) took 1916 ms. Trace:
I20260709 13:15:20.740861 13365 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:20.765913 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:20.743602 13366 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:20.712800 13407 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:20.709267 13393 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:20.741459 13369 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:20.752869 12068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 435) took 1573 ms (client timeout 1612 ms). Trace:
W20260709 13:15:20.806774 12068 rpcz_store.cc:269] 0709 13:15:19.179132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.245051 (+ 65919us) service_pool.cc:224] Handling call
0709 13:15:20.752677 (+1507626us) spinlock_profiling.cc:243] Waited 79.9 ms on lock 0x7b540077fbd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:20.752801 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183671168}
I20260709 13:15:20.730826 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:20.739791 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:20.808192 11443 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:20.708503 13042 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:15:20.720508 13396 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:20.723941 13138 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:20.708804 13301 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.743140 13404 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:20.748217 13247 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:20.779779 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:20.785508 13413 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:20.786170 13392 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:15:20.787427 13072 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.771221 13365 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:20.756906 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:20.801613 13229 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:20.757121 13347 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:20.780368 13326 consensus_meta.cc:287] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 13:15:20.756835 13315 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:20.871160 13315 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.870908 13365 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:20.872879 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.873016 13365 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:20.872679 13302 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.726078 13408 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:20.875989 13408 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:20.852015 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 211489us
W20260709 13:15:20.847673 13336 consensus_meta.cc:287] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.007s
I20260709 13:15:20.846513 13404 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:20.845916 11439 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:20.876168 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 856) took 1014 ms. Trace:
I20260709 13:15:20.759965 13357 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:20.878083 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 250684us
I20260709 13:15:20.878226 13302 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:20.879200 13422 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:20.880074 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 216478us
I20260709 13:15:20.847038 13413 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.881562 13131 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:20.883351 12065 rpcz_store.cc:276] 0709 13:15:19.861350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.861483 (+   133us) service_pool.cc:224] Handling call
0709 13:15:19.861855 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:19.862163 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:19.862188 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:20.022659 (+160471us) log.cc:844] Serialized 26 byte log entry
0709 13:15:20.231645 (+208986us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:20.231690 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.231797 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:20.875911 (+644114us) raft_consensus.cc:1685] finished
0709 13:15:20.875939 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.876097 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13291,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":124,"thread_start_us":201454,"threads_started":1,"wal-append.queue_time_us":476198}
I20260709 13:15:20.846071 13407 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:20.897065 12064 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:20.863678 13347 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:20.881846 13326 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:20.882462 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.898870 13418 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:15:20.888739 13408 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:15:20.882958 13404 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:20.882776 13420 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:15:20.888649 13316 consensus_meta.cc:287] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.632s	user 0.005s	sys 0.005s
I20260709 13:15:20.900805 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:20.926460 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:20.887562 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:20.855513 13072 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:20.857229 13348 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:20.932825 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45381.8us
I20260709 13:15:20.933076 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:20.934123 11511 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:20.934638 13247 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.935310 13369 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:20.935135 13247 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.935956 13369 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:20.936249 13408 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:20.936704 13408 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:20.937078 13247 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:20.937633 13247 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:20.937458 13408 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:20.938658 13408 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:20.944010 13362 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:20.893893 13335 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.944407 13362 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:20.945197 13362 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.945829 13335 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:20.925449 13420 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:20.947073 13420 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.948057 13420 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:20.948621 13326 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:20.949608 13162 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:20.950322 13162 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:20.950750 13162 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:20.951135 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:20.951949 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:20.952348 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.952754 13407 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:20.913166 13131 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.954211 13404 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.933611 13316 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:20.936323 13369 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.766798 11438 rpcz_store.cc:276] 0709 13:15:18.824114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.843626 (+1019512us) service_pool.cc:224] Handling call
0709 13:15:19.942758 (+ 99132us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:20.062800 (+120042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:20.066104 (+  3304us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:20.113101 (+ 46997us) log.cc:844] Serialized 26 byte log entry
0709 13:15:20.367906 (+254805us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:20.371635 (+  3729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.371712 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:20.739932 (+368220us) raft_consensus.cc:1685] finished
0709 13:15:20.739952 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.740124 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":214912,"threads_started":1,"wal-append.queue_time_us":389821}
I20260709 13:15:20.897634 13413 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:20.891328 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:21.040966 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 149713us
W20260709 13:15:21.043063 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.885924 13336 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:20.954989 13131 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:20.948124 13301 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:20.888074 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:21.064778 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.065019 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 177005us
I20260709 13:15:21.065892 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:21.066179 13353 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:21.066763 11440 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:20.951072 13162 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:21.066601 13353 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:21.070655 13353 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:21.070757 13347 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:21.072969 13347 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:21.075518 13415 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:20.993171 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:21.068344 11335 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:21.076903 13383 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.574s	user 0.003s	sys 0.000s
W20260709 13:15:21.077214 13419 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:15:21.077833 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:20.955485 11511 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:21.008551 13318 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:20.943912 13402 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:15:21.081890 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 88781.9us
I20260709 13:15:20.997025 11519 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:20.997145 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63644us
I20260709 13:15:21.083985 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:21.001794 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:20.935871 13418 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:21.014230 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 234524us
I20260709 13:15:21.016613 11513 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:15:20.988512 13247 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:21.024668 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:21.025457 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:21.027434 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.086474 12065 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 1.
I20260709 13:15:21.028631 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.087352 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:21.038317 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.088092 12068 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
W20260709 13:15:21.038501 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 112124us
I20260709 13:15:21.089200 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:21.089498 12069 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 2: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:21.089654 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:21.021976 13392 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.048193 13335 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:21.052804 13138 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.058205 13072 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:21.060752 13388 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.440s	user 0.002s	sys 0.000s
I20260709 13:15:21.026758 11439 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:21.063915 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.040026 13369 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:20.996009 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.091776 13131 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:21.093212 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:21.081043 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:21.093943 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:21.017485 13362 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:21.094338 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:21.110857 11511 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:21.096341 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:21.102743 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.109623 13430 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:21.111871 13430 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.112208 13430 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:21.085228 13418 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:21.112340 13404 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.112583 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.113219 11438 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:21.113097 13418 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.114148 13418 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:21.114993 11439 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:21.105091 13392 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.089226 13425 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:21.118842 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.096155 12061 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:21.119278 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:21.084897 13407 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:21.119875 11510 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:21.105945 13301 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:15:21.120347 13407 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:21.078640 13304 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:21.080801 13109 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:21.103667 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:21.103425 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 425) took 1574 ms. Trace:
I20260709 13:15:21.086059 13421 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.104938 13336 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.096735 13335 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:21.108079 11436 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:21.090134 13314 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:21.119938 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.147490 11442 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:21.151968 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.123478 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:21.124678 11408 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:21.125124 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:21.105824 13344 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:21.167433 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44055.3us
I20260709 13:15:21.098752 13365 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:21.138674 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:21.100215 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:21.154646 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2063) took 2330 ms. Trace:
W20260709 13:15:21.162299 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.172528 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47945.7us
I20260709 13:15:21.121281 13407 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:21.176137 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:21.137738 13411 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 13:15:21.137643 13341 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 1.909s	user 0.000s	sys 0.004s
I20260709 13:15:21.184855 11437 rpcz_store.cc:276] 0709 13:15:18.824581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.857298 (+1032717us) service_pool.cc:224] Handling call
0709 13:15:19.875658 (+ 18360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.665996 (+790338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.154407 (+488411us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b54000fb4d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:21.154581 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811424384}
I20260709 13:15:21.191010 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:21.209231 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:21.136307 13268 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:21.193493 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:21.187649 13421 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:21.215155 13336 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:21.216395 13421 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:21.175861 13431 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:21.196661 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1672) took 1011 ms. Trace:
I20260709 13:15:21.217293 11517 rpcz_store.cc:276] 0709 13:15:20.185074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.632227 (+447153us) service_pool.cc:224] Handling call
0709 13:15:20.680906 (+ 48679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:20.681154 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:20.681202 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:20.698296 (+ 17094us) log.cc:844] Serialized 26 byte log entry
0709 13:15:21.089250 (+390954us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:21.089285 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.089347 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.196485 (+107138us) raft_consensus.cc:1685] finished
0709 13:15:21.196501 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.196611 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":341285,"threads_started":1,"wal-append.queue_time_us":451607}
I20260709 13:15:21.138018 13432 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:15:21.150902 13318 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.200086 11519 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:21.200690 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:21.174786 13141 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:21.206511 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:21.163781 12063 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:21.191846 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.232307 11368 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:15:21.208235 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 110030us
I20260709 13:15:21.211647 12066 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:21.173228 13335 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:21.213338 11364 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:21.135347 13138 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:21.235842 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 131991us
I20260709 13:15:21.183928 13434 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:21.141412 13259 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:21.123219 12062 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:21.137739 13347 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:21.154156 13162 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.223809 11441 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:21.225163 11370 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:21.172896 13344 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:21.184304 13094 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.230764 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:21.232764 13141 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.233675 12033 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:21.163456 13072 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:15:21.237721 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:21.207006 13404 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:21.239838 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:21.122547 13240 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.270787 13240 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.273015 13240 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.213007 13268 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:21.175204 13258 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:15:21.175288 13304 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:21.252434 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 174669us
I20260709 13:15:21.233053 11369 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:21.275530 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.275579 13075 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:21.276387 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:21.276893 13304 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:21.276784 13075 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:21.277112 13408 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:21.277730 13408 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:21.278085 13408 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:21.278836 13075 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:21.226152 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.168498 11514 rpcz_store.cc:276] 0709 13:15:19.528718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.953781 (+425063us) service_pool.cc:224] Handling call
0709 13:15:20.016508 (+ 62727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:20.029598 (+ 13090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:20.029629 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:20.030111 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:15:20.338066 (+307955us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:20.351910 (+ 13844us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.352128 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.352220 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.076652 (+724432us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b2400261158. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b97979c4 00007f35b9795acb 00007f35b9950dc9 00007f35b9950a55 00007f35b995009d 00007f35bc58210e 00007f35bc59988c 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221
0709 13:15:21.080637 (+  3985us) raft_consensus.cc:1685] finished
0709 13:15:21.093821 (+ 13184us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.103338 (+  9517us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8668,"prepare.queue_time_us":7876,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":17386,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":13487,"spinlock_wait_cycles":14303104,"thread_start_us":263600,"threads_started":1,"wal-append.queue_time_us":397169}
I20260709 13:15:21.243294 13347 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:21.280704 13431 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:21.281172 13431 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:21.281103 13347 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:21.257016 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.286372 11364 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:21.286505 13408 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.220742 13314 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:15:21.287869 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.157795 13247 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.288820 13344 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.290118 13344 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:21.290747 13408 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.270221 13426 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:21.293355 13426 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.238330 13348 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:21.294272 13348 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:21.294775 13348 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:21.291292 13437 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:21.291734 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1669) took 1107 ms. Trace:
I20260709 13:15:21.292469 13413 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.222354 13316 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:15:21.293772 13426 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:21.296136 11518 rpcz_store.cc:276] 0709 13:15:20.184365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.493433 (+309068us) service_pool.cc:224] Handling call
0709 13:15:20.493759 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:20.494100 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:20.494129 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:20.511390 (+ 17261us) log.cc:844] Serialized 26 byte log entry
0709 13:15:20.809705 (+298315us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:20.809744 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.809813 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.137131 (+327318us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:21.291435 (+154304us) raft_consensus.cc:1685] finished
0709 13:15:21.291470 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.291675 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4476,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":7893,"spinlock_wait_cycles":5407488,"thread_start_us":233463,"threads_started":1,"wal-append.queue_time_us":357331}
I20260709 13:15:21.295660 13336 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:15:21.281637 13406 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:15:21.299908 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:21.312623 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 81948us
W20260709 13:15:21.297986 13386 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.863s	user 0.000s	sys 0.004s
I20260709 13:15:21.268467 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:21.354100 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 120245us
I20260709 13:15:21.348017 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.349587 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:21.356247 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.311723 11515 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:21.281590 13431 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:21.268887 13141 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:21.233933 13353 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.219326 13430 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.370859 13415 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:21.371695 11367 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:21.371451 13415 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:21.372418 13240 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:21.372831 13240 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:21.372983 13430 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.373204 13432 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:21.373616 13415 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:21.374637 11363 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:21.375667 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:21.376376 12064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 862) took 1513 ms (client timeout 1593 ms). Trace:
I20260709 13:15:21.377424 11364 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:21.366173 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:21.370354 13353 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.378334 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.376709 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:21.381129 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1651) took 2914 ms. Trace:
W20260709 13:15:21.381167 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 81334.8us
W20260709 13:15:21.381459 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 80108.9us
I20260709 13:15:21.352092 13428 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:21.269088 13094 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:21.376785 12064 rpcz_store.cc:269] 0709 13:15:19.862964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.863649 (+   685us) service_pool.cc:224] Handling call
0709 13:15:20.151602 (+287953us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:21.376174 (+1224572us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b5400599054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:21.376318 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697490176}
I20260709 13:15:21.386868 13033 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:15:21.468137 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.552156 13033 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:21.553120 13033 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:21.469168 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.383824 13434 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.384109 13404 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:21.387305 11512 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:15:21.506283 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 119818us
I20260709 13:15:21.309719 13318 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:15:21.384692 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:21.510782 13335 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:21.512100 13347 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.550943 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1790) took 2412 ms. Trace:
I20260709 13:15:21.557492 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:21.378511 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.588343 12060 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:21.588972 13347 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.589285 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.590075 12060 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:21.587781 11444 rpcz_store.cc:276] 0709 13:15:19.138800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.060859 (+922059us) service_pool.cc:224] Handling call
0709 13:15:20.061213 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:20.069222 (+  8009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:20.069251 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:20.074838 (+  5587us) log.cc:844] Serialized 26 byte log entry
0709 13:15:20.193376 (+118538us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:20.193414 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.193507 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.295120 (+1101613us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:21.513289 (+218169us) raft_consensus.cc:1685] finished
0709 13:15:21.513309 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.550846 (+ 37537us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7623,"raft.queue_time_us":15930,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"spinlock_wait_cycles":86411648,"thread_start_us":86952,"threads_started":1,"wal-append.queue_time_us":296194}
W20260709 13:15:21.594480 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.596432 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.598564 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.599133 11444 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:21.599951 11437 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:21.600911 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:21.602185 13050 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:21.603300 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.604487 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.604959 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.605103 11439 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:21.606098 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:21.606868 11441 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:21.606196 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.608474 12065 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:21.611797 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:21.612982 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.613946 11333 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:21.615089 13318 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:21.615693 12953 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:21.512723 13348 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.382884 13240 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:21.383347 11516 rpcz_store.cc:276] 0709 13:15:18.448508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.449062 (+   554us) service_pool.cc:224] Handling call
0709 13:15:18.496178 (+ 47116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.706871 (+210693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:18.706903 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:18.758758 (+ 51855us) log.cc:844] Serialized 26 byte log entry
0709 13:15:19.027840 (+269082us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:19.027876 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.027941 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.090308 (+2062367us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:21.354690 (+264382us) raft_consensus.cc:1685] finished
0709 13:15:21.354708 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.363194 (+  8486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4670,"raft.queue_time_us":324979,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":4452,"spinlock_wait_cycles":16631680,"thread_start_us":389720,"threads_started":2,"wal-append.queue_time_us":288833}
I20260709 13:15:21.304383 13393 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.576530 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:21.577994 11510 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:21.580638 13132 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:21.581832 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:21.560760 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 178468us
I20260709 13:15:21.384460 13428 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:21.952791 11368 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:21.950068 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.773550 11438 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:21.961097 11367 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:21.862252 13431 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.963336 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:21.964074 11363 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:21.964339 13431 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:21.964787 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.965107 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.965685 11368 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:21.966460 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:21.834388 13109 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:21.955866 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 398432us
I20260709 13:15:21.968798 11511 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:21.969185 11409 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:21.841367 12067 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:21.616089 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.967614 11483 connection.cc:256] RPC call timeout handler was delayed by -1.9201329819998136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 13:15:21.971302 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:21.946079 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:21.972184 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.973155 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.973691 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.974967 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:21.967229 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:21.975561 11436 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:21.666074 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.716879 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 142217us
I20260709 13:15:21.977977 11484 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:21.957976 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.862922 13443 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:15:21.983878 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:21.946928 13446 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:15:21.987382 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.781919 13362 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.000s
W20260709 13:15:21.805459 13369 consensus_meta.cc:287] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.680s	user 0.007s	sys 0.001s
I20260709 13:15:21.996529 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.969700 13357 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:21.615473 13318 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:21.801843 11362 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:22.001590 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:21.982344 13259 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.741s	user 0.008s	sys 0.000s
I20260709 13:15:22.002358 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:22.002978 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:22.019990 11516 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:22.020210 11364 raft_consensus.cc:2410] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:21.980916 13094 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.003618 11361 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:21.800350 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:21.968195 13109 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:22.000612 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:22.006186 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:22.007819 11368 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:21.980204 13072 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.003813 13444 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:21.998227 13336 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:21.853371 13141 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:21.996618 13395 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:22.031023 11517 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
I20260709 13:15:22.031427 11370 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:22.021595 13162 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.004235 13430 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:22.010150 13318 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:22.026938 13418 consensus_meta.cc:287] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.912s	user 0.011s	sys 0.000s
I20260709 13:15:22.009290 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:22.013883 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:21.998845 13431 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:21.573694 13434 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.055179 11364 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:22.028095 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:22.020509 13347 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:15:22.063084 11516 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:22.037420 13444 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:22.027714 13406 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:15:22.040696 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:22.063385 13240 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.041683 13072 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.022006 13369 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.066848 11512 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:22.033704 13109 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.084471 13444 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.079164 12062 raft_consensus.cc:2368] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4.
I20260709 13:15:22.072834 13162 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:22.058815 11513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.040s	sys 0.070s
I20260709 13:15:22.076186 13393 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.067006 13430 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.080160 13384 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:22.031059 13428 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.057873 13395 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.134294 12069 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:22.414242 11334 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.148s (SENT) [suppressed 26 similar messages]
W20260709 13:15:22.369316 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.096657 13162 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.431241 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 560) took 1331 ms. Trace:
I20260709 13:15:22.434674 12069 rpcz_store.cc:276] 0709 13:15:21.099702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.092969 (+993267us) service_pool.cc:224] Handling call
0709 13:15:22.114519 (+ 21550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.430968 (+316449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.431175 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:22.433133 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.412536 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:22.285161 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 307212us
I20260709 13:15:22.436600 11445 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:22.096879 11513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.041s	sys 0.071s
W20260709 13:15:22.438606 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:22.439061 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:22.358117 13426 consensus_meta.cc:287] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.982s	user 0.006s	sys 0.004s
I20260709 13:15:22.431551 13109 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.430612 13409 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:22.449471 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 464179us
I20260709 13:15:22.378170 13347 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.286582 13444 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:22.414125 13430 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.371559 11442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:15:22.456323 12069 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:15:22.461854 12069 pending_rounds.cc:85] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 5
I20260709 13:15:22.462092 12069 pending_rounds.cc:107] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:15:22.462325 12069 raft_consensus.cc:2889] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:22.462975 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 562) took 1362 ms. Trace:
I20260709 13:15:22.111804 13384 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:22.296902 13138 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:22.338512 13348 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.438757 13162 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:22.463280 12069 rpcz_store.cc:276] 0709 13:15:21.100122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.448196 (+1348074us) service_pool.cc:224] Handling call
0709 13:15:22.455957 (+  7761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.462746 (+  6789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.462921 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.438351 13439 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:22.468545 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.461572 11442 raft_consensus.cc:606] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:22.472193 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 57978.2us
W20260709 13:15:22.473457 11515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 443) took 1434 ms (client timeout 1504 ms). Trace:
I20260709 13:15:22.473734 11335 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:22.479985 11436 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
W20260709 13:15:22.481983 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.475637 13456 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:15:22.484270 13455 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.484807 13455 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:22.485381 13392 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.487223 13455 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.488023 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1798) took 1756 ms. Trace:
I20260709 13:15:22.488401 11440 rpcz_store.cc:276] 0709 13:15:20.731336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.733260 (+  1924us) service_pool.cc:224] Handling call
0709 13:15:20.733593 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.067130 (+333537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.487793 (+1420663us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54001126d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:22.487964 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765439872}
I20260709 13:15:22.489655 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:22.490137 13297 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.490254 11440 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:22.490736 13297 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.460048 13440 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:22.476338 11515 rpcz_store.cc:269] 0709 13:15:21.025986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.045670 (+ 19684us) service_pool.cc:224] Handling call
0709 13:15:22.459916 (+1414246us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54002a76d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:22.460044 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437284608}
I20260709 13:15:22.483841 12069 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:15:22.452411 13422 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.530s	user 0.096s	sys 0.121s
I20260709 13:15:22.495466 11266 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39460 (request call id 363) took 6588 ms. Trace:
I20260709 13:15:22.491398 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:22.499873 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 564) took 1399 ms. Trace:
I20260709 13:15:22.502030 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:22.502938 12069 rpcz_store.cc:276] 0709 13:15:21.100412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.474916 (+1374504us) service_pool.cc:224] Handling call
0709 13:15:22.483618 (+  8702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.499632 (+ 16014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.499832 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.496953 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:22.492924 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 329) took 1390 ms. Trace:
I20260709 13:15:22.483189 13439 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.505753 13268 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.514647 13268 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:22.515388 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:22.515792 13439 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:22.493018 13440 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:22.517962 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.518812 13079 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:22.514004 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:22.498669 13430 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:15:22.499089 13138 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.521323 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:22.521775 13138 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.492005 13072 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:22.506510 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:15:22.523119 12061 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:22.523204 13430 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:22.523672 13430 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:22.524004 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:22.519322 13079 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.453327 13141 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:22.505633 11438 rpcz_store.cc:276] 0709 13:15:21.102746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.113899 (+ 11153us) service_pool.cc:224] Handling call
0709 13:15:21.114201 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.961415 (+847214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.492663 (+531248us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b5400315bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:22.492862 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016095616}
W20260709 13:15:22.526050 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.524461 13430 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.531519 13413 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.458747 13409 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:22.464881 13033 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.534737 11440 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:22.523119 11369 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:22.508587 13431 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:22.522035 12066 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:22.524210 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:22.531879 13430 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:22.502614 13422 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.582s	user 0.096s	sys 0.121s
I20260709 13:15:22.546851 11445 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:22.521178 13395 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.457274 13426 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.664551 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1492) took 1939 ms. Trace:
I20260709 13:15:22.532317 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 438) took 1504 ms. Trace:
I20260709 13:15:22.495733 13455 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:22.552119 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 101327us
W20260709 13:15:22.532675 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59023.7us
I20260709 13:15:22.665884 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:22.510238 13452 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:15:22.664964 11366 rpcz_store.cc:276] 0709 13:15:20.725347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.725767 (+   420us) service_pool.cc:224] Handling call
0709 13:15:21.614800 (+889033us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:21.614968 (+   168us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:21.615296 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:21.615322 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:21.971838 (+356516us) log.cc:844] Serialized 26 byte log entry
0709 13:15:22.432540 (+460702us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:22.432582 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.432644 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.641739 (+209095us) raft_consensus.cc:1685] finished
0709 13:15:22.641761 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.664472 (+ 22711us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2169,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":55995264,"thread_start_us":355733,"threads_started":1,"wal-append.queue_time_us":484455}
I20260709 13:15:22.502936 13347 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.668594 11444 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:22.472730 13457 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:22.523888 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 565) took 1423 ms. Trace:
I20260709 13:15:22.660591 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2080) took 1665 ms. Trace:
I20260709 13:15:22.502318 13369 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.675701 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 568) took 1574 ms. Trace:
I20260709 13:15:22.676102 12066 rpcz_store.cc:276] 0709 13:15:21.101093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.661682 (+1560589us) service_pool.cc:224] Handling call
0709 13:15:22.675149 (+ 13467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.675332 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.675359 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.675429 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:22.675465 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.675518 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.675641 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.502436 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:22.673097 11445 rpcz_store.cc:276] 0709 13:15:20.995025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.995166 (+   141us) service_pool.cc:224] Handling call
0709 13:15:22.412337 (+1417171us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:22.660533 (+248196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14513024}
I20260709 13:15:22.679661 11436 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:22.543015 13421 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:22.495595 13297 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.666189 13426 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:22.508960 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 573) took 1340 ms. Trace:
I20260709 13:15:22.708943 12066 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:22.507887 13434 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:22.661175 13439 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:22.716193 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 569) took 1611 ms. Trace:
I20260709 13:15:22.716655 12066 rpcz_store.cc:276] 0709 13:15:21.101252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.708509 (+1607257us) service_pool.cc:224] Handling call
0709 13:15:22.708790 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.711503 (+  2713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.712948 (+  1445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.671434 13406 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:22.716907 11483 connection.cc:256] RPC call timeout handler was delayed by -1.759932981999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2389s for a response
I20260709 13:15:22.718127 12066 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:22.741127 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 570) took 1639 ms. Trace:
I20260709 13:15:22.741473 12066 rpcz_store.cc:276] 0709 13:15:21.101418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.717709 (+1616291us) service_pool.cc:224] Handling call
0709 13:15:22.717970 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.740874 (+ 22904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.741064 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.742643 12066 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:22.693382 11440 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:15:22.739823 11445 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:22.745642 11445 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:22.748334 11440 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:22.743149 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 571) took 1641 ms. Trace:
I20260709 13:15:22.748950 12066 rpcz_store.cc:276] 0709 13:15:21.101572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.742193 (+1640621us) service_pool.cc:224] Handling call
0709 13:15:22.742468 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.742950 (+   482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.743102 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.750303 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 863) took 1505 ms. Trace:
I20260709 13:15:22.750578 12066 rpcz_store.cc:276] 0709 13:15:21.244680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.749601 (+1504921us) service_pool.cc:224] Handling call
0709 13:15:22.749840 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.750029 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.750054 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.750091 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:22.750123 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.750172 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.750257 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:22.752004 12066 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:22.754010 11440 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:22.759527 12953 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:22.709777 13431 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.665349 13455 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.712803 13369 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.641896 13033 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.758057 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:22.762717 13268 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:22.776223 12953 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.677532 13348 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.712731 11365 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:22.681288 13457 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.778710 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:22.670073 13422 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:22.660610 13318 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:22.740273 13440 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.503310 11266 rpcz_store.cc:276] 0709 13:15:15.906971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.907117 (+   146us) service_pool.cc:224] Handling call
0709 13:15:22.495355 (+6588238us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:20.621150 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:20.692790 (+ 71640us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:20.692825 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:15:20.692842 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:20.704539 (+ 11697us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:20.708235 (+  3696us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:20.709781 (+  1546us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:20.749627 (+ 39846us) tablet.cc:701] Row locks acquired
0709 13:15:20.749664 (+    37us) write_op.cc:260] PREPARE: finished
0709 13:15:20.765911 (+ 16247us) write_op.cc:270] Start()
0709 13:15:20.771444 (+  5533us) write_op.cc:276] Timestamp: P: 1783602920763532 usec, L: 0
0709 13:15:20.779530 (+  8086us) op_driver.cc:348] REPLICATION: starting
0709 13:15:20.893847 (+114317us) log.cc:844] Serialized 530 byte log entry
0709 13:15:21.252641 (+358794us) op_driver.cc:464] REPLICATION: finished
0709 13:15:21.842875 (+590234us) write_op.cc:301] APPLY: starting
0709 13:15:21.854819 (+ 11944us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:21.861825 (+  7006us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:21.889204 (+ 27379us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:21.949143 (+ 59939us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:21.998600 (+ 49457us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:15:22.005842 (+  7242us) write_op.cc:312] APPLY: finished
0709 13:15:22.023871 (+ 18029us) log.cc:844] Serialized 23 byte log entry
0709 13:15:22.038911 (+ 15040us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:22.045271 (+  6360us) write_op.cc:454] Released schema lock
0709 13:15:22.053610 (+  8339us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:22.072503 (+ 18893us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:22.074053 (+  1550us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41914,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":527984,"apply.run_cpu_time_us":2521,"apply.run_wall_time_us":252066,"num_ops":1,"prepare.queue_time_us":237823,"prepare.run_cpu_time_us":23601,"prepare.run_wall_time_us":482174,"raft.queue_time_us":107193,"raft.run_cpu_time_us":50109,"raft.run_wall_time_us":134587,"replication_time_us":462642,"thread_start_us":477168,"threads_started":4,"wal-append.queue_time_us":208008}]]}
I20260709 13:15:22.757203 13314 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.762007 13297 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.779369 13072 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:22.786690 13268 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.740865 13439 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.672124 12061 rpcz_store.cc:276] 0709 13:15:21.100628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.522636 (+1422008us) service_pool.cc:224] Handling call
0709 13:15:22.522922 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.523643 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.523836 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:22.797436 11365 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1488) took 2092 ms (client timeout 1972 ms). Trace:
I20260709 13:15:22.806649 11363 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:22.759881 13406 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.712622 12062 rpcz_store.cc:276] 0709 13:15:21.168634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.311456 (+142822us) service_pool.cc:224] Handling call
0709 13:15:22.508744 (+1197288us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54005d6e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:22.508895 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108376448}
I20260709 13:15:22.716495 13409 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.811264 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:22.784549 13431 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.799443 13440 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:22.816963 11441 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:22.819430 12953 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.785066 13369 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.821300 11440 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:22.665155 11513 rpcz_store.cc:276] 0709 13:15:21.021220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.152585 (+131365us) service_pool.cc:224] Handling call
0709 13:15:21.152861 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:21.269750 (+116889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:21.269793 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:21.363665 (+ 93872us) log.cc:844] Serialized 26 byte log entry
0709 13:15:22.455184 (+1091519us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:22.457193 (+  2009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.465255 (+  8062us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.472977 (+  7722us) raft_consensus.cc:1685] finished
0709 13:15:22.480109 (+  7132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.525819 (+ 45710us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":151135,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":45501,"thread_start_us":1040432,"threads_started":2,"wal-append.queue_time_us":570226}
I20260709 13:15:22.739475 13384 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:22.811296 13336 consensus_meta.cc:287] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.767s	user 0.007s	sys 0.003s
I20260709 13:15:22.794144 13348 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.800777 13297 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:22.901152 11441 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:22.809394 13268 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:22.872684 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:22.896066 11445 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:22.803249 13314 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:22.843232 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:22.821768 13072 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:22.817234 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 151431us
I20260709 13:15:22.875279 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:22.914736 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:22.809902 13432 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:15:22.810437 11365 rpcz_store.cc:269] 0709 13:15:20.702436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.703775 (+  1339us) service_pool.cc:224] Handling call
0709 13:15:22.472592 (+1768817us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:22.794388 (+321796us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11782016}
W20260709 13:15:22.903020 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 57215.5us
W20260709 13:15:22.918146 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 75026.6us
I20260709 13:15:22.918917 11334 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:22.824091 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 280178us
I20260709 13:15:22.894393 13336 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:22.841933 11519 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:22.911006 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:22.907153 11441 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:22.807341 13442 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:15:22.817006 13347 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.872402 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:22.854660 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 559) took 1755 ms. Trace:
I20260709 13:15:22.894877 13384 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:22.894075 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:22.929558 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:22.949361 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:22.944301 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.929107 12060 rpcz_store.cc:276] 0709 13:15:21.099483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.074575 (+975092us) service_pool.cc:224] Handling call
0709 13:15:22.074914 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.075987 (+  1073us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:22.459909 (+383922us) log.cc:844] Serialized 14 byte log entry
0709 13:15:22.797109 (+337200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.806280 (+  9171us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:22.811716 (+  5436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.816887 (+  5171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.854590 (+ 37703us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":435971,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":6261,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":2144,"thread_start_us":601146,"threads_started":2,"wal-append.queue_time_us":431239}
I20260709 13:15:22.915906 13033 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:22.895069 13348 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:22.972788 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 53945.6us
I20260709 13:15:22.973551 11334 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:22.917217 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:22.932418 11514 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:22.975960 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:22.915877 13464 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:22.936694 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:22.976279 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 53676.9us
W20260709 13:15:23.001750 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 86813.8us
W20260709 13:15:22.977988 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 56571.2us
W20260709 13:15:23.004279 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.005774 12066 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:22.931988 13406 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.008862 11444 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:22.933224 13469 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:22.949349 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:22.975550 13033 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:23.002663 13357 consensus_meta.cc:287] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.988s	user 0.010s	sys 0.000s
I20260709 13:15:23.021951 13466 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:23.022465 13407 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:23.022861 13407 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:23.007285 13432 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:22.976583 13444 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.001s
I20260709 13:15:22.931689 13072 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 13:15:23.017845 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 81231.7us
I20260709 13:15:23.019501 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.023201 13407 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:22.922206 13422 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.024037 13469 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:15:23.026393 13162 consensus_meta.cc:287] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.000s
I20260709 13:15:22.976655 13472 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:22.957906 13417 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.096628 13162 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:23.097337 13417 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.098369 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.027508 11443 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:23.099205 12062 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:23.027364 13473 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:23.028820 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 566) took 1928 ms. Trace:
I20260709 13:15:23.029330 13455 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.030376 13297 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.005949 13468 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:23.006209 13138 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:15:23.005347 13141 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:15:23.032184 13439 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:23.032804 13431 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.010349 13418 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:22.976150 13456 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:15:23.038950 13430 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:23.039693 13457 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:23.080147 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:23.011693 13336 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.032141 13318 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.026717 12066 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:23.044050 13432 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:23.006448 13471 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.092s
I20260709 13:15:23.082942 13268 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.277014 13079 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:23.312522 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:23.159313 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 141566us
W20260709 13:15:23.323561 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 242915us
I20260709 13:15:23.350067 11436 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:23.346894 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:23.356663 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:23.308768 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.190133 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:23.357915 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.358297 12068 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:23.359633 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.355266 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:23.278832 11409 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.753s (SENT)
I20260709 13:15:23.185717 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 554) took 2087 ms. Trace:
I20260709 13:15:23.188374 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1337) took 2233 ms. Trace:
I20260709 13:15:23.349581 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:23.360366 11437 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:23.362668 11511 rpcz_store.cc:276] 0709 13:15:20.955383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.142628 (+187245us) service_pool.cc:224] Handling call
0709 13:15:21.968633 (+826005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.969172 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.188175 (+1219003us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54001509d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:23.188314 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1898864896}
I20260709 13:15:23.363699 11364 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:23.100041 12069 rpcz_store.cc:276] 0709 13:15:21.100795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.527378 (+1426583us) service_pool.cc:224] Handling call
0709 13:15:22.641000 (+113622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.642861 (+  1861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.642890 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:22.666436 (+ 23546us) log.cc:844] Serialized 26 byte log entry
0709 13:15:22.890625 (+224189us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:22.890730 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.890885 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:23.028618 (+137733us) raft_consensus.cc:1685] finished
0709 13:15:23.028636 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.028766 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1433,"raft.queue_time_us":18777,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":223612,"threads_started":1,"wal-append.queue_time_us":334800}
W20260709 13:15:23.348176 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 268115us
W20260709 13:15:23.358314 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.361560 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.366847 12066 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:23.160372 13109 consensus_meta.cc:287] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.700s	user 0.002s	sys 0.007s
I20260709 13:15:23.373682 11369 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:23.031322 13369 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.361934 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:23.086216 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.407342 12061 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:23.408542 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.409343 12069 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:23.365833 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:23.370146 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 360230us
I20260709 13:15:23.372742 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:23.409966 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.410118 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.410524 11364 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.369443 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 555) took 2270 ms. Trace:
I20260709 13:15:23.362015 12067 rpcz_store.cc:276] 0709 13:15:21.098459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.098591 (+   132us) service_pool.cc:224] Handling call
0709 13:15:21.841128 (+742537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.972540 (+131412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.185485 (+1212945us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b54005d5f50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:23.185644 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1674579840}
I20260709 13:15:23.279181 13357 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:23.279040 13344 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.385674 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:23.385898 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 107092us
I20260709 13:15:23.360625 13431 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:23.423066 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.386184 11362 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:23.386428 13406 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:23.386911 13138 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.428478 13138 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.429708 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.430317 11444 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.387334 12066 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:23.388784 13268 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.436117 11436 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:15:23.359793 13297 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:23.437299 13297 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:23.437390 11444 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:23.438244 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.438215 13297 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.350632 13418 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.440834 13418 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.389726 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.391341 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:23.391805 13469 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:23.394325 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:23.385834 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.382762 13455 consensus_queue.cc:1059] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:15:23.474787 13455 consensus_queue.cc:1243] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:23.475332 13455 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:23.475678 13455 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:23.457906 13431 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.476562 13455 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.477746 13455 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.478359 13431 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:23.401954 11363 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 1989) took 1437 ms (client timeout 1543 ms). Trace:
I20260709 13:15:23.479717 13407 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.480782 11442 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:23.481812 13407 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.482169 11442 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:23.457544 11439 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:23.403112 13240 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.187446 13353 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:23.405169 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.251215 11365 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:23.264113 13428 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:23.355679 13449 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.004s	sys 0.013s
I20260709 13:15:23.349366 13430 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:23.374691 13439 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:23.412483 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46735.4us
I20260709 13:15:23.503428 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:23.413451 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:23.503791 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.383214 11437 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:23.413975 12068 rpcz_store.cc:276] 0709 13:15:21.098754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.985031 (+886277us) service_pool.cc:224] Handling call
0709 13:15:23.189945 (+1204914us) spinlock_profiling.cc:243] Waited 961 us on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:23.369382 (+179437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2210560}
I20260709 13:15:23.415581 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.505570 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.506748 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.265489 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:23.507791 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.346254 13316 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:23.417568 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45580.3us
I20260709 13:15:23.124101 13472 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:23.418645 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 225408us
I20260709 13:15:23.411132 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.159627 13464 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:23.386396 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.458453 13417 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.424379 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.522570 12067 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.427384 12064 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:23.436254 13050 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:23.422326 13404 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:23.318163 13395 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:15:23.442032 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.446513 12069 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:23.447798 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.536406 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.502235 13439 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:23.419286 13457 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:23.545172 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.358836 13336 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.422142 13357 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:23.472792 13384 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.000s
I20260709 13:15:23.473526 11513 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:23.474135 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:23.479449 11363 rpcz_store.cc:269] 0709 13:15:21.964238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.964567 (+   329us) service_pool.cc:224] Handling call
0709 13:15:23.401762 (+1437195us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54002db754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:23.401892 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934530944}
I20260709 13:15:23.529088 13335 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.419828 13314 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.489322 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.545657 13456 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:23.441061 13426 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:23.336674 13413 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.346015 13476 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.511706 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:23.514387 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 101032us
I20260709 13:15:23.600452 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:15:23.502511 13449 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.004s	sys 0.013s
I20260709 13:15:23.601118 13395 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:15:23.424746 11409 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 1.972s (SENT)
I20260709 13:15:23.602268 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:23.488997 13353 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:23.449398 13344 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.514509 13240 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.508188 11512 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.501089 13258 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.521719 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.534026 11442 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:23.537380 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.450085 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.633601 11444 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:23.537751 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:23.583971 13426 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:23.537796 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:23.634588 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 96859.3us
W20260709 13:15:23.634745 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 211010us
W20260709 13:15:23.635633 11409 leader_election.cc:336] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.992s (SENT)
I20260709 13:15:23.512648 13444 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.636005 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:23.509027 13463 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:23.544556 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.638338 11363 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:23.638446 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:23.639465 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:23.536144 13404 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.412055 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.512220 13348 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:23.514424 13406 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.511674 13472 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:23.502375 13418 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:15:23.691450 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55262.2us
I20260709 13:15:23.552213 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.461150 13432 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.553761 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.485656 11364 raft_consensus.cc:2410] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:23.528658 13050 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:23.554385 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.572366 11440 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:23.546164 13318 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:23.574165 13439 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.553176 12061 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:23.547717 13457 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.576454 11516 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:15:23.489892 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 73143.9us
I20260709 13:15:23.546737 13469 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:23.578676 12067 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:23.538579 13362 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:15:23.538379 13407 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:15:23.515102 13316 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.575623 11514 raft_consensus.cc:2410] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:23.553105 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 556) took 2454 ms. Trace:
I20260709 13:15:23.599016 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:23.514912 12953 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:23.518404 13369 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:23.575564 13384 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:23.548017 13422 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:15:23.630270 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.709323 12953 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:23.709530 13384 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.709822 12953 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.629810 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.711700 13384 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.600009 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:23.634130 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 96463.9us
I20260709 13:15:23.712630 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:23.713020 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.640851 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:23.641409 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.551666 13430 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.714346 11441 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:23.714799 13430 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.715107 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.715636 11441 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.693229 11445 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:23.717190 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.718123 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.718626 11445 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
W20260709 13:15:23.694634 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 59019.1us
I20260709 13:15:23.719666 12953 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.720847 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:23.722122 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:23.693915 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:23.697093 12063 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:23.697928 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.641918 13314 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:23.704346 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.641817 13444 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.642045 13456 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:23.603650 13413 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.704964 13495 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:23.625722 13268 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:23.695025 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:23.773269 11518 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:15:23.695014 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 56619us
I20260709 13:15:23.697927 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:23.776024 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.776690 11516 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:23.774569 11334 leader_election.cc:400] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4.
I20260709 13:15:23.727592 11369 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:23.692675 13471 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:23.738960 11511 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:23.634155 13240 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.699143 13431 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:23.745653 11441 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:23.741010 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.766248 11444 raft_consensus.cc:2368] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5.
I20260709 13:15:23.548357 13478 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.196s
I20260709 13:15:23.767503 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.690135 13463 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.699572 13318 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.703919 13440 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:23.705613 13417 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:23.705988 13457 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.711850 11437 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:23.641866 13258 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.705178 13229 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.783780 12953 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.708658 13247 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:23.697279 13348 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.698383 13406 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.801465 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:23.817462 12034 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:23.770925 13314 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:23.862087 13314 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:23.801978 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 89437.3us
I20260709 13:15:23.863029 11484 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:23.631417 13464 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.729154 11438 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:23.803719 13409 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:23.727177 13469 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.769213 13456 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.738495 13384 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:23.814927 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.766125 13426 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.727564 12064 rpcz_store.cc:276] 0709 13:15:21.098954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.000243 (+901289us) service_pool.cc:224] Handling call
0709 13:15:23.427012 (+1426769us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:23.427211 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.534106 (+106895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.551977 (+ 17871us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8541056}
I20260709 13:15:23.816286 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:23.804392 13268 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:23.853552 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:23.813383 13471 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.857039 13505 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:23.858156 13501 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:15:23.809191 13050 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.859300 13240 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.860054 13344 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.765043 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:23.799852 13498 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:23.809834 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:23.624004 13476 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:23.927048 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2078) took 2966 ms (client timeout 2077 ms). Trace:
W20260709 13:15:23.927594 11443 rpcz_store.cc:269] 0709 13:15:20.960341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.001520 (+ 41179us) service_pool.cc:224] Handling call
0709 13:15:23.926861 (+2925341us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b5400438ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:23.926966 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4452794624}
I20260709 13:15:23.917937 11444 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:23.848444 13449 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:23.931265 13449 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:23.932237 13449 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.816107 13417 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:23.935982 13417 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.936481 13348 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.849134 13440 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:23.937641 13440 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:23.938097 13469 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.876631 11436 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:23.847558 13490 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.881875 11437 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:23.940022 13500 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:15:23.936635 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.941329 11365 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:23.941074 11437 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:23.864066 13247 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.942307 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.943818 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.864553 13499 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:15:23.813998 11514 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:23.942312 13268 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.953598 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:23.954941 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:23.867204 13437 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.955662 13437 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.956449 13138 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.958166 13437 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.864797 13494 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:23.959309 12063 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:23.913690 13407 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:15:23.959578 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.960263 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.960711 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.961148 13335 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:23.940984 13050 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:23.961591 12063 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:23.906463 12068 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:23.962510 12067 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
W20260709 13:15:23.906633 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 167550us
I20260709 13:15:23.871438 13422 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:23.910864 11517 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:23.867352 13468 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.911878 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:23.872334 13456 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:23.852921 13473 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:15:23.849507 13488 consensus_queue.cc:1059] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:15:23.915299 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:23.915315 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:23.915829 13508 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:23.901186 13486 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
I20260709 13:15:23.855947 13229 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.929929 13463 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:23.930421 13258 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:23.930477 13471 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.935842 13318 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.899982 13357 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:23.945122 12066 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:23.881803 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 107944us
I20260709 13:15:23.916877 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:23.939971 13490 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:23.914300 13426 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:24.357081 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 443) took 1115 ms. Trace:
I20260709 13:15:24.362291 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1339) took 1530 ms. Trace:
W20260709 13:15:24.248246 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.375892 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:23.963227 12068 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:15:24.012993 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:24.378058 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:24.378602 12033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.002s	sys 0.016s
W20260709 13:15:24.379009 12033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.002s	sys 0.016s
W20260709 13:15:24.377942 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 462690us
I20260709 13:15:23.964375 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:24.378692 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 463435us
I20260709 13:15:24.265523 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:24.381325 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:24.380681 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:24.382920 12066 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:24.383056 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.062958 11444 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:24.383867 11370 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
W20260709 13:15:24.350363 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 487376us
I20260709 13:15:24.384379 12066 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:15:24.374483 11366 rpcz_store.cc:276] 0709 13:15:23.241403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.241845 (+   442us) service_pool.cc:224] Handling call
0709 13:15:23.242171 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.448051 (+205880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.448081 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.448121 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:23.448150 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.351804 (+903654us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400249354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:24.356878 (+  5074us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.357018 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473042176}
I20260709 13:15:24.385269 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 465) took 1397 ms. Trace:
I20260709 13:15:24.385319 11516 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:24.380247 11437 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:24.385712 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.392130 11370 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:24.392346 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.385794 12066 rpcz_store.cc:276] 0709 13:15:23.001576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.383924 (+1382348us) service_pool.cc:224] Handling call
0709 13:15:24.384196 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:24.385054 (+   858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.385222 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.379293 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 823883us
I20260709 13:15:24.392802 11364 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:24.359726 13472 consensus_meta.cc:287] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.660s	user 0.004s	sys 0.004s
I20260709 13:15:24.394681 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.380542 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 463) took 1405 ms. Trace:
I20260709 13:15:24.390323 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:24.365273 13428 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.863s	user 0.007s	sys 0.002s
I20260709 13:15:24.393410 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:24.387705 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:24.409653 11363 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:24.381587 13499 raft_consensus.cc:2764] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:15:24.380867 11439 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.004s	sys 0.029s
I20260709 13:15:24.364164 13498 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:23.959519 13444 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:24.400842 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.242713 13421 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:24.434705 12068 rpcz_store.cc:276] 0709 13:15:22.985472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.962744 (+977272us) service_pool.cc:224] Handling call
0709 13:15:23.963014 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:24.380309 (+417295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.380489 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.454411 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:23.961302 12061 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 1.
W20260709 13:15:24.436872 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46593us
I20260709 13:15:24.270800 13476 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:24.320740 13314 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.388772 13138 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:24.460992 13138 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.393328 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:24.098152 11510 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:24.458667 11370 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:24.438908 11366 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:24.377843 13422 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:24.451680 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:24.045208 13478 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.452579 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:24.454463 13050 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:24.483278 11516 raft_consensus.cc:2410] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:23.961585 13335 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:24.436430 13437 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:24.241969 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 459) took 1267 ms. Trace:
I20260709 13:15:24.388589 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.433107 13384 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:24.453876 13072 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:24.281567 13430 consensus_meta.cc:287] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.003s
W20260709 13:15:24.455555 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 70486.8us
I20260709 13:15:24.437119 13472 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:24.457969 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:24.436781 12066 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:24.464408 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 71157us
I20260709 13:15:24.391598 13494 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:24.019976 13404 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:24.466720 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:24.398102 13162 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:24.393220 13508 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:24.452126 11439 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.004s	sys 0.029s
I20260709 13:15:24.374537 11515 rpcz_store.cc:276] 0709 13:15:22.832233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.832381 (+   148us) service_pool.cc:224] Handling call
0709 13:15:23.441647 (+609266us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b5800010b98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:23.441832 (+   185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:23.500332 (+ 58500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.500367 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:23.500901 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:15:23.952200 (+451299us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:23.952243 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.952311 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:24.362009 (+409698us) raft_consensus.cc:1685] finished
0709 13:15:24.362031 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.362230 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41734,"raft.queue_time_us":60303,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":6758,"spinlock_wait_cycles":3286784,"thread_start_us":424226,"threads_started":1,"wal-append.queue_time_us":749119}
I20260709 13:15:24.394232 13455 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:23.925585 13505 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.388708 13501 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:15:24.537838 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:24.538514 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:24.619995 12068 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
W20260709 13:15:24.467021 11512 consensus_meta.cc:287] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.000s
I20260709 13:15:24.389209 13466 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:15:24.454921 13421 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:24.576964 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:24.420830 11483 connection.cc:256] RPC call timeout handler was delayed by -1.3824674280001545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
I20260709 13:15:24.468735 13362 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:24.582244 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 124366us
I20260709 13:15:24.631124 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:24.632004 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:24.568166 12063 rpcz_store.cc:276] 0709 13:15:22.974533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.961108 (+986575us) service_pool.cc:224] Handling call
0709 13:15:23.961401 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.962058 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.241903 (+279845us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.568749 13478 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.402760 13468 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:24.568092 13449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.017s	sys 0.059s
W20260709 13:15:24.611881 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 145229us
I20260709 13:15:24.578096 13072 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:24.588871 13384 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:24.588862 13409 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:15:24.653090 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 76222.7us
W20260709 13:15:24.591432 13457 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.000s
I20260709 13:15:24.620958 11516 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
W20260709 13:15:24.625033 12069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 870) took 1609 ms (client timeout 1958 ms). Trace:
I20260709 13:15:24.589075 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 558) took 3478 ms. Trace:
W20260709 13:15:24.628543 11483 connection.cc:256] RPC call timeout handler was delayed by -1.3554674280001109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
W20260709 13:15:24.657033 11483 connection.cc:256] RPC call timeout handler was delayed by -1.3365674280000803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
I20260709 13:15:24.589675 13499 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:24.616333 13437 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:24.592051 12033 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:24.644486 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:24.617794 13455 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.592577 13494 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:24.590279 13050 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:24.471896 13413 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:24.653046 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1823) took 1121 ms. Trace:
I20260709 13:15:24.623361 11512 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
W20260709 13:15:24.656122 12069 rpcz_store.cc:269] 0709 13:15:23.015483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.409893 (+394410us) service_pool.cc:224] Handling call
0709 13:15:24.624862 (+1214969us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b54000587d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:24.624975 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82376576}
W20260709 13:15:24.648567 13449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.020s	sys 0.059s
I20260709 13:15:24.733636 13468 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:24.737004 13496 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.039s	user 0.025s	sys 0.025s
I20260709 13:15:24.636402 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 343) took 1175 ms. Trace:
I20260709 13:15:24.645421 13348 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:24.656909 12065 rpcz_store.cc:276] 0709 13:15:21.099287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.000511 (+901224us) service_pool.cc:224] Handling call
0709 13:15:22.023848 (+ 23337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.024097 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.024125 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:23.238065 (+1213940us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b2c000a6788. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:23.238456 (+   391us) log.cc:844] Serialized 26 byte log entry
0709 13:15:23.422320 (+183864us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:23.556657 (+134337us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.698914 (+142257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.698968 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:24.577512 (+878544us) raft_consensus.cc:1685] finished
0709 13:15:24.577600 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.577740 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15511,"prepare.queue_time_us":128931,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1733,"raft.queue_time_us":79625,"raft.run_cpu_time_us":8034,"raft.run_wall_time_us":61395,"spinlock_wait_cycles":12477184,"thread_start_us":403886,"threads_started":3,"wal-append.queue_time_us":278526}
I20260709 13:15:24.698563 11436 rpcz_store.cc:276] 0709 13:15:23.531721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.550291 (+ 18570us) service_pool.cc:224] Handling call
0709 13:15:24.652854 (+1102563us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54001b6cd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:24.652975 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422867712}
W20260709 13:15:24.698239 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44361us
I20260709 13:15:24.650774 13498 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:15:24.701687 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 109734us
W20260709 13:15:24.702840 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70906.7us
I20260709 13:15:24.705952 13512 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:24.707448 13384 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.716662 13472 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:24.717229 13508 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:24.719887 13505 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:24.622449 13422 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.626327 13335 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.723227 11444 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:24.730700 13464 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:24.659336 13440 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:24.658077 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:24.665063 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:24.738018 11439 rpcz_store.cc:276] 0709 13:15:23.460804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.482877 (+ 22073us) service_pool.cc:224] Handling call
0709 13:15:23.483174 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.483351 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:23.636748 (+153397us) log.cc:844] Serialized 14 byte log entry
0709 13:15:24.621262 (+984514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.621335 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:24.621395 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.621461 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.633902 (+ 12441us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":283201,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":3979,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":7789,"spinlock_wait_cycles":38784,"thread_start_us":1060760,"threads_started":2,"wal-append.queue_time_us":922774}
I20260709 13:15:24.742923 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:24.777694 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.743814 12033 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:24.755069 11366 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
W20260709 13:15:24.755915 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 106526us
I20260709 13:15:24.665817 13437 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.758452 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:24.669656 13409 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.769464 12065 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:24.703037 13455 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.665848 13515 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:24.657622 13457 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:24.772933 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 106439us
I20260709 13:15:24.783138 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:24.773845 13464 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.786363 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:24.788239 13472 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:24.756407 11512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1340) took 1921 ms (client timeout 1956 ms). Trace:
I20260709 13:15:24.784440 13505 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.790467 13437 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:24.828368 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:24.829633 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:24.829777 13348 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:24.830165 11442 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:24.830705 13348 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.832219 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1825) took 1300 ms. Trace:
I20260709 13:15:24.832588 11445 rpcz_store.cc:276] 0709 13:15:23.532205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.719156 (+186951us) service_pool.cc:224] Handling call
0709 13:15:24.830802 (+1111646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.830968 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.830991 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.831031 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:24.831060 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.831107 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.832060 (+   953us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002797d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:24.832164 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2556004736}
I20260709 13:15:24.833309 13464 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:24.759140 13404 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:15:24.681401 13421 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.799075 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.801970 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.808753 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:24.789280 13510 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:15:24.816030 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:24.774441 13440 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:24.765849 13525 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:15:24.822634 11368 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:24.768420 13490 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:24.798342 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.825489 13050 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.825980 13504 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:24.826443 12062 raft_consensus.cc:2410] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:24.800135 13409 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:24.826824 11438 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2097) took 1449 ms (client timeout 1609 ms). Trace:
I20260709 13:15:24.799685 13141 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:24.835067 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:24.791807 13512 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:24.789605 13162 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:24.798553 11370 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:24.792626 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 134774us
I20260709 13:15:24.797964 13422 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:24.800853 13335 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:24.913949 11511 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:15:24.914506 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 131500us
I20260709 13:15:24.862288 11515 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:24.914527 11367 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:24.915302 11366 raft_consensus.cc:2410] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:15:24.924948 13504 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:24.915226 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 129005us
W20260709 13:15:24.915548 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 118276us
I20260709 13:15:24.916728 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.961776 11363 raft_consensus.cc:2393] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:24.823232 13094 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:24.921749 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.922567 11444 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:24.922955 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.923780 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:24.800215 13515 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:24.816368 13488 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:24.929276 11445 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:15:24.819698 11512 rpcz_store.cc:269] 0709 13:15:22.832544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.839449 (+  6905us) service_pool.cc:224] Handling call
0709 13:15:23.265365 (+425916us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b5800010b98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9eee 0000000000375939 00007f35c194d1e0 00007f35c1a160c4 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:24.754247 (+1488882us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16879872}
W20260709 13:15:24.938958 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 103988us
I20260709 13:15:24.893919 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:24.919730 13421 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:24.941632 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:24.941092 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:24.952140 11483 leader_election.cc:336] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.691s (SENT)
I20260709 13:15:24.952935 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:24.954216 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.919304 13505 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:24.914603 11364 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:24.956283 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.863575 13498 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:24.939641 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.891094 13408 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:15:24.964964 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:24.823843 13472 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:24.962518 13162 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:24.940825 13456 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:24.942015 13362 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.182337 11368 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:24.972157 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:25.183168 11367 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:24.998888 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1694) took 1372 ms. Trace:
I20260709 13:15:24.998219 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1703) took 1254 ms. Trace:
I20260709 13:15:25.191478 11515 rpcz_store.cc:276] 0709 13:15:23.743796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.619790 (+875994us) service_pool.cc:224] Handling call
0709 13:15:24.997983 (+378193us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540031ef54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:24.998148 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449326848}
I20260709 13:15:25.186025 11442 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:25.191272 11511 rpcz_store.cc:276] 0709 13:15:23.626166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.758186 (+1132020us) service_pool.cc:224] Handling call
0709 13:15:24.998716 (+240530us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400109354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:24.998833 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296569728}
W20260709 13:15:25.186631 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.186728 11335 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:25.202425 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:25.085624 13529 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:15:25.187171 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:25.013382 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1350) took 2140 ms. Trace:
I20260709 13:15:25.187726 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:25.267765 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 315648us
W20260709 13:15:25.266386 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63014.5us
W20260709 13:15:25.268826 11483 connection.cc:256] RPC call timeout handler was delayed by -1.2753674279999814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:15:24.984426 11517 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1691) took 1458 ms (client timeout 1808 ms). Trace:
W20260709 13:15:25.269413 11517 rpcz_store.cc:269] 0709 13:15:23.525921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.572460 (+ 46539us) service_pool.cc:224] Handling call
0709 13:15:24.984215 (+1411755us) spinlock_profiling.cc:243] Waited 95.3 ms on lock 0x7b540014ffd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:24.984355 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219162496}
I20260709 13:15:25.189296 13536 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:25.189467 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 247938us
I20260709 13:15:25.265870 11445 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:25.270490 11519 rpcz_store.cc:276] 0709 13:15:22.873043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.929283 (+ 56240us) service_pool.cc:224] Handling call
0709 13:15:25.013118 (+2083835us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400064fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:25.013302 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3761644416}
I20260709 13:15:25.277010 11366 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:24.954487 13141 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:25.274507 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:24.990537 11268 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.280061 11444 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.093114 13404 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:24.994974 13072 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:25.140323 13314 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.141355 13247 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:24.965508 13094 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:25.286559 11268 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 3, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.277902 13476 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.290995 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:25.190092 13109 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.190267 13407 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.303393 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:25.263687 13297 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:25.197991 13515 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.267836 13521 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:25.173048 13478 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.263180 13495 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:24.927477 11438 rpcz_store.cc:269] 0709 13:15:23.377061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.389437 (+ 12376us) service_pool.cc:224] Handling call
0709 13:15:24.826624 (+1437187us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54002797d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:24.826764 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562882304}
I20260709 13:15:25.281006 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.267694 11439 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:25.347157 11408 leader_election.cc:336] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.716s (SENT)
I20260709 13:15:25.313640 11268 catalog_manager.cc:5697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 3, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.092334 13455 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.263803 13505 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:25.287899 13404 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:25.311268 13072 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.276285 13464 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:15:25.264669 13437 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.342061 11515 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:25.316780 13094 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.333192 13521 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.312215 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:25.346273 11444 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:25.347791 11408 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:25.363618 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:25.317335 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:25.364373 11515 raft_consensus.cc:2376] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5. Re-sending same reply.
W20260709 13:15:25.348981 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 162331us
I20260709 13:15:25.333750 11514 raft_consensus.cc:2368] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5.
I20260709 13:15:25.000061 13440 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.323269 13297 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:25.328835 13314 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:25.334287 13457 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:25.365226 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.374756 11515 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:24.997350 13494 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.366348 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:25.340855 13409 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.375969 13494 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:25.380152 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.380972 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1700) took 1677 ms. Trace:
I20260709 13:15:25.381307 11515 rpcz_store.cc:276] 0709 13:15:23.703450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.379866 (+1676416us) service_pool.cc:224] Handling call
0709 13:15:25.380917 (+  1051us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.382144 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.383159 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.388149 11515 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:15:25.327668 13109 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.327889 13407 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.361717 13348 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:25.362793 13468 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.398993 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:25.402076 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 127649us
I20260709 13:15:25.367455 13521 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.402575 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:25.399585 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:25.403378 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:25.405512 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 114603us
I20260709 13:15:25.406206 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:25.374522 11512 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.412681 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 355) took 1035 ms. Trace:
I20260709 13:15:25.374276 13314 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.419751 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:25.401441 13348 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.409708 13544 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:25.409839 13409 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.412011 13455 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.412762 13407 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.413556 13472 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:25.414757 13545 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:25.353571 13498 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.392081 13525 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:25.401688 13468 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.408464 11408 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:25.416617 11436 rpcz_store.cc:276] 0709 13:15:24.377375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.742603 (+365228us) service_pool.cc:224] Handling call
0709 13:15:24.754228 (+ 11625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:24.913150 (+158922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.913193 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:24.914121 (+   928us) log.cc:844] Serialized 26 byte log entry
0709 13:15:25.204496 (+290375us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:25.204533 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.204604 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:25.412438 (+207834us) raft_consensus.cc:1685] finished
0709 13:15:25.412459 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.412625 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2721,"raft.queue_time_us":48021,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":421159,"threads_started":2,"wal-append.queue_time_us":384542}
I20260709 13:15:25.403378 13476 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:25.462523 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54119.2us
I20260709 13:15:25.435807 13478 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:25.424335 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:25.463215 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:25.425007 11335 leader_election.cc:400] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5.
I20260709 13:15:25.466542 11335 leader_election.cc:403] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:15:25.409052 13508 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:25.363039 13072 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.422834 13348 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:25.407884 13457 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.403919 13141 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.341773 13495 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.467640 13476 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:25.475678 13551 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5.
I20260709 13:15:25.485257 13472 raft_consensus.cc:1064] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:25.312208 13504 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.509093 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 883) took 1117 ms. Trace:
I20260709 13:15:25.510931 13472 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.494266 13316 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.495512 13406 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:25.511991 11515 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:25.512671 13072 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.497274 13258 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:25.513118 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 348) took 1880 ms. Trace:
W20260709 13:15:25.501786 12062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 869) took 2492 ms (client timeout 2662 ms). Trace:
I20260709 13:15:25.407084 13404 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.440346 13547 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:25.439160 13297 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:25.371557 13534 consensus_queue.cc:1059] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:15:25.423075 11365 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:15:25.421818 13314 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:25.460988 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 130818us
I20260709 13:15:25.397101 13408 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.468096 13162 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:25.469260 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44985.3us
W20260709 13:15:25.470122 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 47113.8us
W20260709 13:15:25.514993 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 554409us
I20260709 13:15:25.459800 13545 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.472000 13464 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.515759 13395 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:25.564049 13408 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:25.564368 13395 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:25.464011 13468 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.556731 13456 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.465186 13544 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:25.565601 13544 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:25.552096 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2092) took 2205 ms. Trace:
I20260709 13:15:25.535686 11511 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:25.436668 13409 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:25.556241 13517 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.920s	user 0.009s	sys 0.000s
I20260709 13:15:25.563964 12033 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:25.567973 13404 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:25.676751 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1354) took 2311 ms. Trace:
I20260709 13:15:25.513675 13437 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.553870 13336 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:25.671494 13258 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:25.669085 13481 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:15:25.672519 13395 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.556602 13478 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:25.564186 13545 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:25.520424 13109 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:25.558085 13422 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.000s
W20260709 13:15:25.567927 11518 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1701) took 1860 ms (client timeout 2046 ms). Trace:
W20260709 13:15:25.745371 11518 rpcz_store.cc:269] 0709 13:15:23.707219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.393773 (+686554us) service_pool.cc:224] Handling call
0709 13:15:25.567744 (+1173971us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b5400065ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:25.567871 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147888896}
I20260709 13:15:25.746630 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.747334 11518 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:25.748281 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.749346 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.532107 13505 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:25.678755 13468 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.509546 12061 rpcz_store.cc:276] 0709 13:15:24.391494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.738263 (+346769us) service_pool.cc:224] Handling call
0709 13:15:24.738643 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.738872 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:24.788975 (+ 50103us) log.cc:844] Serialized 14 byte log entry
0709 13:15:25.508564 (+719589us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18003aa140. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:15:25.508724 (+   160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.508779 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:25.508831 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.508901 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.509030 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1330,"prepare.queue_time_us":186561,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2994,"raft.queue_time_us":558475,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":46,"spinlock_wait_cycles":5261568,"thread_start_us":186214,"threads_started":2,"wal-append.queue_time_us":388114}
I20260709 13:15:25.564627 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:25.516124 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 882) took 1125 ms. Trace:
I20260709 13:15:25.522140 13495 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:25.674057 12067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 879) took 2631 ms (client timeout 2242 ms). Trace:
W20260709 13:15:25.515588 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 591874us
I20260709 13:15:25.566401 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:25.738181 13409 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.506690 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.762395 11519 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:25.762820 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 464) took 1098 ms. Trace:
I20260709 13:15:25.763118 11519 rpcz_store.cc:276] 0709 13:15:24.664265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.506410 (+842145us) service_pool.cc:224] Handling call
0709 13:15:25.762768 (+256358us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:25.521337 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:25.763967 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:25.765522 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 200942us
I20260709 13:15:25.513602 11441 rpcz_store.cc:276] 0709 13:15:23.632458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.855551 (+223093us) service_pool.cc:224] Handling call
0709 13:15:24.653355 (+797804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.653531 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.653558 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.653601 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:24.653631 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.653706 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.510865 (+857159us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54001b6cd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:25.510993 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813257472}
I20260709 13:15:25.568836 11440 rpcz_store.cc:276] 0709 13:15:23.346563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.346662 (+    99us) service_pool.cc:224] Handling call
0709 13:15:25.551913 (+2205251us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400398d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:25.552032 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463585664}
W20260709 13:15:25.724978 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 161095us
I20260709 13:15:25.725749 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 594) took 1348 ms. Trace:
I20260709 13:15:25.767585 12068 rpcz_store.cc:276] 0709 13:15:24.377095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.627430 (+250335us) service_pool.cc:224] Handling call
0709 13:15:25.674129 (+1046699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.674307 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:25.674331 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.674367 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:25.674396 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.674446 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.725553 (+ 51107us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400509950. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:25.725686 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406325120}
I20260709 13:15:25.728241 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 446) took 2290 ms. Trace:
I20260709 13:15:25.560544 13138 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:25.774458 12065 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:25.513717 12062 rpcz_store.cc:269] 0709 13:15:23.013197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.712743 (+699546us) service_pool.cc:224] Handling call
0709 13:15:25.501561 (+1788818us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54005d6954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:25.501718 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2474125696}
I20260709 13:15:25.724215 11510 rpcz_store.cc:276] 0709 13:15:23.364923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.368909 (+  3986us) service_pool.cc:224] Handling call
0709 13:15:23.387038 (+ 18129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.473906 (+1086868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.676497 (+1202591us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b5400064ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:25.676684 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538823040}
I20260709 13:15:25.730767 13512 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:25.502985 13540 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:15:25.534516 11517 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:15:25.733141 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.563835 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:25.556740 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:25.786031 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.563865 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:25.741554 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1831) took 2101 ms. Trace:
W20260709 13:15:25.788726 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 224928us
W20260709 13:15:25.782461 13421 consensus_meta.cc:287] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.000s
I20260709 13:15:25.564626 13408 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.552716 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:25.732184 13258 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:25.528580 13335 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.000s
I20260709 13:15:25.749888 11518 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:25.532693 13439 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.479130 13353 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.678239 13534 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.502702 13432 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.541381 11362 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:25.546237 11513 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:25.756045 12069 rpcz_store.cc:276] 0709 13:15:24.391039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.731710 (+340671us) service_pool.cc:224] Handling call
0709 13:15:24.732126 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:24.732531 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.732558 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:24.733027 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:15:24.860779 (+127752us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:25.192796 (+332017us) log.cc:844] Serialized 14 byte log entry
0709 13:15:25.515823 (+323027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.515894 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:25.515934 (+    40us) raft_consensus.cc:1685] finished
0709 13:15:25.515948 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.516067 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":322773,"prepare.queue_time_us":97937,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":323246,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":187,"thread_start_us":400362,"threads_started":2,"wal-append.queue_time_us":592011}
I20260709 13:15:25.556354 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:25.760680 12067 rpcz_store.cc:269] 0709 13:15:23.042887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.704420 (+661533us) service_pool.cc:224] Handling call
0709 13:15:25.673839 (+1969419us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400509954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:25.673976 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369559936}
I20260709 13:15:25.761138 11437 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:25.569239 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:25.786697 13499 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:15:25.786900 13138 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:15:25.768989 11515 rpcz_store.cc:276] 0709 13:15:23.438119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.388770 (+1950651us) service_pool.cc:224] Handling call
0709 13:15:25.389062 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.512856 (+123794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.727940 (+215084us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400371254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:25.728171 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282244480}
I20260709 13:15:25.777757 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:25.753949 13535 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:25.778524 11333 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:25.492393 13079 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:25.779469 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:25.779556 11483 connection.cc:256] RPC call timeout handler was delayed by -1.0858122490001278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
I20260709 13:15:25.782316 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:25.783206 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:25.783790 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:25.737329 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 449) took 2298 ms. Trace:
I20260709 13:15:25.790218 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 447) took 2351 ms. Trace:
I20260709 13:15:25.788563 11443 rpcz_store.cc:276] 0709 13:15:23.640335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.928525 (+288190us) service_pool.cc:224] Handling call
0709 13:15:23.928840 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:23.946820 (+ 17980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.946854 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:24.020264 (+ 73410us) log.cc:844] Serialized 26 byte log entry
0709 13:15:24.467512 (+447248us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:24.650497 (+182985us) log.cc:844] Serialized 14 byte log entry
0709 13:15:24.653493 (+  2996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.653566 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:25.555672 (+902106us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:25.732378 (+176706us) raft_consensus.cc:1685] finished
0709 13:15:25.741339 (+  8961us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.741490 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20342,"prepare.queue_time_us":263966,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":36154,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":51344,"spinlock_wait_cycles":28254976,"thread_start_us":381206,"threads_started":2,"wal-append.queue_time_us":328859}
I20260709 13:15:25.469630 13384 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:25.789517 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:25.791213 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 227441us
I20260709 13:15:25.776633 13362 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.774498 13247 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.566882 13297 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:25.991971 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 209713us
W20260709 13:15:26.060117 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 270655us
I20260709 13:15:26.060761 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:25.797878 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:25.799739 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:25.566460 13544 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.775775 13314 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:25.794636 13437 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.064046 11518 raft_consensus.cc:2376] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4. Re-sending same reply.
I20260709 13:15:26.002081 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:26.070080 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68061.6us
I20260709 13:15:26.070428 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.070735 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:25.504747 13229 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:25.798498 13094 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.800944 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:25.797299 13050 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.968366 11442 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:25.968379 11517 rpcz_store.cc:276] 0709 13:15:23.438341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.533894 (+2095553us) service_pool.cc:224] Handling call
0709 13:15:25.534202 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.789976 (+255774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.790169 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:25.967691 11511 rpcz_store.cc:276] 0709 13:15:23.438783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.535189 (+2096406us) service_pool.cc:224] Handling call
0709 13:15:25.535515 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.737049 (+201534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.737268 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:25.968152 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.085150 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:25.905148 13540 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:15:25.904202 13498 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:25.908432 13464 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.802042 13422 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:25.984599 13499 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:26.095815 13499 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:26.096194 13515 consensus_meta.cc:287] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.755s	user 0.004s	sys 0.007s
I20260709 13:15:25.798760 13478 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.096628 13515 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:25.815843 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.987537 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 1995) took 2348 ms. Trace:
I20260709 13:15:26.062747 12065 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:26.006238 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 227781us
W20260709 13:15:26.102217 13558 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:15:26.103209 13348 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.000s
I20260709 13:15:26.103622 13348 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.104606 13094 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:25.987368 13395 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.107125 13395 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:26.107389 13348 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:25.804225 13417 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.064543 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.065475 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.082747 13335 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.805919 13258 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.066136 13440 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.067559 13535 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:26.073019 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 503822us
W20260709 13:15:26.139384 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.803193 13534 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:26.081969 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.146072 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.146643 11370 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.139441 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.079001 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.968799 13079 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:26.149379 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 178212us
I20260709 13:15:26.091450 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.072427 13247 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:25.991654 13138 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.932024 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:25.967974 13472 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.862258 11512 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:26.062052 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:26.167793 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 105808us
I20260709 13:15:26.098158 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.003513 13384 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:25.988979 13468 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.075312 13316 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:26.172582 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:26.108101 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.185775 11370 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:25.859429 13408 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.091554 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 450) took 2648 ms. Trace:
I20260709 13:15:26.081272 13421 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:26.109508 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.057996 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.080556 13449 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:25.855700 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:25.992537 13409 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:25.860806 13521 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.084105 13050 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:26.206687 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.207252 13050 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.208735 12064 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:26.209854 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.076340 13553 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:26.211229 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.211338 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.211730 13553 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:26.212208 13553 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.212914 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:26.134790 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.213840 12064 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:26.135850 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.215240 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.126236 12065 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:26.143044 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.216053 12064 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:26.217072 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.217168 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.217988 12068 raft_consensus.cc:2410] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:26.143656 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.144801 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:26.219903 13556 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.133s	user 0.000s	sys 0.003s
I20260709 13:15:26.220278 11368 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:26.145455 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.080062 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.148128 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.000635 13353 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.092078 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.224983 12066 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:26.108843 13314 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:26.100430 11362 rpcz_store.cc:276] 0709 13:15:23.638856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.641918 (+  3062us) service_pool.cc:224] Handling call
0709 13:15:24.823813 (+1181895us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:25.540986 (+717173us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9e9c 00007f35b98d9a5b 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:25.541176 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.832287 (+291111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.932147 (+ 99860us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76001920}
I20260709 13:15:26.166281 11369 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:26.125965 13335 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:26.150494 11517 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:15:26.170040 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 71964.5us
I20260709 13:15:26.227982 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.228201 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.137001 13537 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.229082 13537 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.229745 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.230298 13521 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:26.171339 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 111392us
I20260709 13:15:26.069097 13407 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:26.231606 13407 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.179203 12069 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:26.097180 13498 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.231762 13521 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:26.096716 13499 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.155172 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 587) took 2288 ms. Trace:
I20260709 13:15:26.188414 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.242040 11370 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:26.228797 13457 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:26.242933 13457 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.153855 13533 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.247923 13533 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:26.193290 11515 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.249881 11361 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:26.193285 11511 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:26.251328 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.251930 11511 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.253976 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:26.188149 13357 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:26.193066 13421 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.256177 13421 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:26.258723 13444 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:26.176208 13348 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.259866 13348 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:26.260390 13348 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.261082 13476 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:26.176481 13384 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.170490 11510 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:26.264453 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.265298 11510 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:26.266304 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.185750 13316 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.268474 13316 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.271497 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:26.177063 11512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1357) took 1396 ms (client timeout 1716 ms). Trace:
I20260709 13:15:26.177495 13555 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:26.190486 11513 rpcz_store.cc:276] 0709 13:15:23.439006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.545468 (+2106462us) service_pool.cc:224] Handling call
0709 13:15:25.545838 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.006311 (+460473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.087849 (+ 81538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:26.219836 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.227191 13335 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.230819 13314 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:26.151151 13503 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.234632 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:26.235332 13407 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.240499 13498 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:26.241465 11514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 453) took 2752 ms (client timeout 2062 ms). Trace:
I20260709 13:15:26.245699 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.245546 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:26.246459 12064 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.241688 12060 rpcz_store.cc:276] 0709 13:15:23.862236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.568447 (+706211us) service_pool.cc:224] Handling call
0709 13:15:24.629019 (+ 60572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:24.668354 (+ 39335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.704125 (+ 35771us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:24.792121 (+ 87996us) log.cc:844] Serialized 26 byte log entry
0709 13:15:24.957711 (+165590us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:25.273690 (+315979us) log.cc:844] Serialized 14 byte log entry
0709 13:15:25.520746 (+247056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.520837 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:26.124952 (+604115us) raft_consensus.cc:1685] finished
0709 13:15:26.124977 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:26.150467 (+ 25490us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":318614,"prepare.queue_time_us":238818,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":249474,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":19309,"thread_start_us":135467,"threads_started":1,"wal-append.queue_time_us":558945}
I20260709 13:15:26.259156 13444 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:26.196053 13449 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.170014 13472 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.163353 13079 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.292734 12065 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.243296 13457 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.314136 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:26.313411 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.337536 12069 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
W20260709 13:15:26.319490 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 74007.2us
I20260709 13:15:26.338565 11515 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
W20260709 13:15:26.295101 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 60576us
W20260709 13:15:26.318826 11514 rpcz_store.cc:269] 0709 13:15:23.488647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.954696 (+466049us) service_pool.cc:224] Handling call
0709 13:15:26.241204 (+2286508us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400197dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:26.241363 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3047524736}
I20260709 13:15:26.291596 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.341535 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.322774 13454 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:26.341689 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.309121 13316 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.312501 13503 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.311019 13421 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.324287 13537 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:26.325003 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.325274 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1508) took 1523 ms. Trace:
I20260709 13:15:26.326761 13408 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.306893 13314 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:26.243584 13512 consensus_queue.cc:1175] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6e87b213a4554d94825aceec352fc094" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:26.241436 13499 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.292450 13555 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:15:26.286093 11512 rpcz_store.cc:269] 0709 13:15:24.780638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.276527 (+495889us) service_pool.cc:224] Handling call
0709 13:15:26.176969 (+900442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:26.328370 13247 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:26.329722 13545 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.575s	user 0.001s	sys 0.007s
I20260709 13:15:26.330351 13395 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.330720 13449 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.195288 13535 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.351686 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.289988 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:26.337630 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.382776 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.382933 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.383507 11370 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
W20260709 13:15:26.297516 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.385206 13316 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.385752 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.387161 11366 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:26.388800 11364 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:26.389822 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.321882 13444 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.340256 13422 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:26.311049 13335 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.342334 11514 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:26.323477 11511 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:26.342243 13454 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:26.294560 13533 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:26.349423 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.350606 11363 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:26.402654 11333 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:26.403074 13335 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.368196 13545 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:26.403329 13439 consensus_meta.cc:287] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.005s
I20260709 13:15:26.403743 13335 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:26.403820 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:26.404131 13335 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:26.404546 13335 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.404956 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.340927 13138 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:26.406415 13504 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:26.407490 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.408492 13449 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.410005 13553 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.410413 13449 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:26.326663 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.411815 13444 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.412667 12061 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:26.412755 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.413591 13546 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.414352 11365 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:26.322933 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.352335 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:26.307848 13515 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:26.356000 11519 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:26.356106 11518 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:26.356679 12066 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.416587 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.418764 11364 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:26.418699 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.368248 13440 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:15:26.421697 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:26.422672 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:26.361469 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.353703 13432 consensus_meta.cc:287] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.007s
I20260709 13:15:26.326833 12067 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:15:26.365546 13555 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.379678 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.355331 13079 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.431712 11516 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:26.355669 13314 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.197168 13534 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.392848 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.357307 12064 raft_consensus.cc:2376] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7. Re-sending same reply.
I20260709 13:15:26.437148 13247 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:15:26.437539 11516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 458) took 2809 ms (client timeout 2695 ms). Trace:
W20260709 13:15:26.437918 11516 rpcz_store.cc:269] 0709 13:15:23.627772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.655908 (+1028136us) service_pool.cc:224] Handling call
0709 13:15:26.437388 (+1781480us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54000663d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:26.437477 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3819435904}
I20260709 13:15:26.438818 13247 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.439227 13247 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:26.394151 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.356024 13430 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:15:26.399784 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.405048 13545 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:26.321833 13409 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:26.390233 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:26.400748 13404 consensus_meta.cc:287] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.669s	user 0.006s	sys 0.007s
I20260709 13:15:26.368175 13562 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 13:15:26.429875 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 50276.8us
I20260709 13:15:26.462216 11483 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:26.462683 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.463594 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:26.463793 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.429960 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:26.354012 11361 rpcz_store.cc:276] 0709 13:15:24.801949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.819583 (+ 17634us) service_pool.cc:224] Handling call
0709 13:15:26.325099 (+1505516us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400166a54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:26.325217 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279828864}
I20260709 13:15:26.376699 13395 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.394747 11513 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:26.418843 13449 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.389941 13407 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.403869 13511 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.435117 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.432291 13428 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.436030 13534 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.436364 13314 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.473141 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:26.473826 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:26.439392 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 454) took 2941 ms. Trace:
I20260709 13:15:26.421355 13408 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:26.446524 11519 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:15:26.448232 11366 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.379603 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.404662 13472 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:26.424952 13546 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:26.457568 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.457410 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 300726us
I20260709 13:15:26.460836 13430 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:26.432649 13515 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.500058 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.501276 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:26.466725 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:26.433553 13297 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:26.477406 11516 rpcz_store.cc:276] 0709 13:15:23.497607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.438687 (+2941080us) service_pool.cc:224] Handling call
0709 13:15:26.438960 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.439117 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:26.439140 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:26.439180 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:26.439212 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.439260 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:26.439346 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:26.458662 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.465387 13440 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.497452 11408 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:26.497924 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.498210 12069 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:26.502830 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:26.469864 13482 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:26.442848 13050 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.513015 11516 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:26.483855 13432 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.484099 13476 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:26.494551 13430 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:26.515516 11518 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:15:26.504335 11512 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:26.485133 13545 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:26.484268 13422 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:26.519059 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1693) took 2941 ms. Trace:
I20260709 13:15:26.470198 13454 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.505009 13408 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:26.522289 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.557631 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.518093 13440 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.523672 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.524240 13482 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:26.559445 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.533631 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.523315 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.472234 13449 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.535115 12066 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:26.544931 13546 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:26.544665 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 313214us
W20260709 13:15:26.548839 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 294922us
I20260709 13:15:26.548976 12068 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:26.535600 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.508661 11366 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:26.551589 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 457) took 2924 ms. Trace:
I20260709 13:15:26.575613 11518 rpcz_store.cc:276] 0709 13:15:23.627488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.514964 (+2887476us) service_pool.cc:224] Handling call
0709 13:15:26.515280 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:26.551246 (+ 35966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.551512 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:26.573380 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.576540 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.577034 11368 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:15:26.554754 11516 rpcz_store.cc:276] 0709 13:15:23.577984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.509341 (+2931357us) service_pool.cc:224] Handling call
0709 13:15:26.512850 (+  3509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.518816 (+  5966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.519003 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:26.577621 11366 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.485841 13428 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.509130 13472 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.508911 13464 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:26.485999 13511 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.516752 13404 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:26.505476 13297 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.562805 11436 raft_consensus.cc:2410] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:26.518810 13421 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:26.566144 11442 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:26.573719 13422 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.488552 13564 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:26.567507 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.577382 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.569299 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.572746 11363 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.576946 11518 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:15:26.615434 11518 pending_rounds.cc:85] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 1
I20260709 13:15:26.615695 11518 pending_rounds.cc:107] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:15:26.615908 11518 raft_consensus.cc:2889] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:26.616531 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1695) took 2979 ms. Trace:
I20260709 13:15:26.616819 11518 rpcz_store.cc:276] 0709 13:15:23.637308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.576376 (+2939068us) service_pool.cc:224] Handling call
0709 13:15:26.576648 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.616305 (+ 39657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.616470 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:26.617908 11518 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:26.618438 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1698) took 2920 ms. Trace:
W20260709 13:15:26.488654 13141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.025s	sys 0.022s
I20260709 13:15:26.554473 13545 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:26.557958 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:26.598189 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:26.619669 11440 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.567451 13533 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.566754 13537 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.567991 13546 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.519078 13503 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.578908 11513 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.606076 13297 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.573678 13316 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:26.570411 13457 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.535887 13553 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.607024 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:26.615181 11445 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:26.638664 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.618746 11518 rpcz_store.cc:276] 0709 13:15:23.698401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.617468 (+2919067us) service_pool.cc:224] Handling call
0709 13:15:26.617753 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.618222 (+   469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.618390 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:26.630138 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 63016.4us
I20260709 13:15:26.640245 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:26.600421 13472 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.606843 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:26.606750 11366 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:26.640602 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1702) took 2903 ms. Trace:
W20260709 13:15:26.626880 13141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.025s	sys 0.025s
I20260709 13:15:26.642904 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:26.648365 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.648666 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.649216 11445 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:26.613329 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.645866 11518 rpcz_store.cc:276] 0709 13:15:23.736802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.639758 (+2902956us) service_pool.cc:224] Handling call
0709 13:15:26.640132 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.640298 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:26.640323 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:26.640363 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:26.640393 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:26.640444 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:26.640545 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:26.649969 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:26.646518 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70492.5us
I20260709 13:15:26.626791 13545 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:26.658133 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.606243 13478 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:15:26.628890 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.629946 13564 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:26.634377 13521 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.630865 13546 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:26.666976 11517 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:26.619925 13422 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.659343 11365 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:26.664214 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 97923.9us
I20260709 13:15:26.686353 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.637780 13348 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.670162 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:26.692298 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:26.647619 13499 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.693125 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:26.687858 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 221212us
I20260709 13:15:26.694479 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:26.694706 12033 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:26.662469 13395 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:26.633844 13537 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:26.687351 13567 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:15:26.701208 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.728549 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.702566 11368 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.705394 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.730352 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:26.731500 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.729338 11363 raft_consensus.cc:2410] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:26.732503 11370 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:26.631650 13430 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.634778 13533 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.676222 11361 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:26.751853 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:26.697517 11366 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.643926 13316 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.689107 13141 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:26.659862 13555 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.696184 11438 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:26.736950 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:26.663960 13512 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:15:26.683195 13566 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:15:26.684520 13570 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:15:26.706220 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:26.739377 13050 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.689318 13569 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:15:26.741168 13422 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.742101 13564 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.743302 13472 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.745548 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:26.641120 13408 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.723783 13314 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:26.724048 13495 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.748463 13577 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:26.683847 13521 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:26.887980 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 247788us
I20260709 13:15:26.727921 13421 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:27.067207 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.710894 13498 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:27.067847 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:27.076768 11368 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:27.084844 11368 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
W20260709 13:15:26.773752 13384 consensus_meta.cc:287] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.510s	user 0.000s	sys 0.008s
I20260709 13:15:27.068800 11441 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:26.710708 13563 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:26.778291 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 49183.8us
I20260709 13:15:26.756047 11366 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.765008 11437 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:26.769610 13565 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:27.071839 11369 raft_consensus.cc:2410] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:26.727306 13499 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.711371 13571 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:27.073722 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:27.161903 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88273.9us
I20260709 13:15:26.864917 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:26.733404 13545 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:26.910166 13417 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.004s
W20260709 13:15:27.068142 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.096652 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:27.019192 13553 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:27.070816 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.171412 11441 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:27.171473 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.771625 13407 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.068928 13566 consensus_queue.cc:1243] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:26.769942 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:27.113328 13581 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:26.767596 11512 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:27.069077 13430 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.756888 13576 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:26.767071 13457 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.754813 13439 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:27.066329 13511 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.102315 13454 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:27.072525 13353 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.765734 13449 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.764827 13537 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:26.736441 13395 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.360s
I20260709 13:15:27.080670 13534 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:27.079767 13336 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.162240 13565 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:26.729748 13348 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:27.165230 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:27.069438 13141 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:26.759236 13335 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:27.101945 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:27.182337 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 116817us
I20260709 13:15:27.089365 13495 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:26.863982 13482 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.297s
I20260709 13:15:27.088040 13421 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:27.079876 13549 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:15:27.164783 13512 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:15:27.161456 13384 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:27.158879 13563 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:27.106895 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:27.898331 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.764827 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:27.903000 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1526) took 1331 ms. Trace:
I20260709 13:15:27.173748 11369 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:27.174751 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:27.924801 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2128) took 1314 ms. Trace:
I20260709 13:15:27.925207 11441 rpcz_store.cc:276] 0709 13:15:26.610548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.174533 (+563985us) service_pool.cc:224] Handling call
0709 13:15:27.924587 (+750054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:27.926860 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:27.164211 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:27.073828 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 367698us
I20260709 13:15:28.074450 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:26.759745 11440 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:27.558379 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.095041 11440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2124) took 1511 ms (client timeout 1509 ms). Trace:
W20260709 13:15:28.095486 11440 rpcz_store.cc:269] 0709 13:15:26.583695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.597955 (+ 14260us) service_pool.cc:224] Handling call
0709 13:15:28.094833 (+1496878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:27.184994 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:28.097965 11438 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.099702 13515 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.100389 13515 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.102528 13515 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.104344 13478 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:15:27.810158 11513 consensus_meta.cc:287] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 1.164s	user 0.008s	sys 0.008s
I20260709 13:15:28.105316 11513 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
W20260709 13:15:28.106030 11513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1726) took 1630 ms (client timeout 1604 ms). Trace:
I20260709 13:15:27.814989 11439 raft_consensus.cc:2393] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:28.106417 13549 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:28.106988 11439 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1840) took 1420 ms (client timeout 1629 ms). Trace:
I20260709 13:15:28.106787 13407 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:27.830696 11365 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:15:28.105278 13478 consensus_queue.cc:1243] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:27.180197 11514 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:27.840489 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1717) took 2104 ms. Trace:
I20260709 13:15:28.109025 13607 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:28.109361 11510 rpcz_store.cc:276] 0709 13:15:25.735594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.341449 (+605855us) service_pool.cc:224] Handling call
0709 13:15:27.839994 (+1498545us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400197dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:27.840292 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697024896}
I20260709 13:15:28.111330 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:28.112104 11510 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:28.112620 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1732) took 1359 ms. Trace:
I20260709 13:15:27.820474 13440 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:27.316893 13247 consensus_meta.cc:287] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.877s	user 0.003s	sys 0.006s
I20260709 13:15:28.118623 13247 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:27.330077 13297 consensus_meta.cc:287] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.686s	user 0.008s	sys 0.000s
I20260709 13:15:27.353498 13314 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:27.389341 13476 consensus_meta.cc:287] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.837s	user 0.001s	sys 0.008s
W20260709 13:15:27.704581 13544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.009s	sys 0.007s
W20260709 13:15:28.131340 13544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.664s	user 0.011s	sys 0.008s
W20260709 13:15:28.132395 11514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1370) took 1779 ms (client timeout 2049 ms). Trace:
W20260709 13:15:28.132786 11514 rpcz_store.cc:269] 0709 13:15:26.353149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.412282 (+ 59133us) service_pool.cc:224] Handling call
0709 13:15:28.132018 (+1719736us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b540014bed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.132255 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574675840}
I20260709 13:15:28.133349 13247 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:27.392597 13546 consensus_meta.cc:287] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.708s	user 0.008s	sys 0.000s
I20260709 13:15:28.137522 13546 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:27.350453 13587 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.177s	user 0.002s	sys 0.000s
I20260709 13:15:27.528293 13348 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.142175 13348 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.142539 13546 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:27.386452 13404 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.783s	user 0.009s	sys 0.000s
I20260709 13:15:27.538712 13141 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.143970 11438 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:27.344395 13316 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:27.557794 13335 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.579849 13336 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.150187 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:28.151015 11514 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:27.532527 13454 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:27.562247 13384 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.565748 13421 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:27.550913 13563 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:27.428215 13432 consensus_meta.cc:287] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.892s	user 0.007s	sys 0.003s
I20260709 13:15:27.570050 13439 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:27.579010 13576 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:27.181062 13571 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:27.193759 13581 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:27.330725 13583 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 13:15:27.172341 13584 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:27.391647 13555 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:27.182258 13553 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:27.382381 13545 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.802707 13535 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:27.331018 13533 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.399291 13521 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.392899 13499 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:27.556764 13495 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.531467 13482 consensus_queue.cc:1243] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:27.536726 13593 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:27.376454 13594 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.869349 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 360) took 1301 ms. Trace:
W20260709 13:15:27.878218 11516 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.011s	sys 0.022s
I20260709 13:15:27.191993 12063 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 13:15:27.894901 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.012s	sys 0.016s
W20260709 13:15:27.905275 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.913988 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 362) took 1344 ms. Trace:
I20260709 13:15:27.917538 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:27.199467 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.903354 11362 rpcz_store.cc:276] 0709 13:15:26.571144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.572670 (+  1526us) service_pool.cc:224] Handling call
0709 13:15:26.764707 (+192037us) spinlock_profiling.cc:243] Waited 7.08 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9eee 0000000000375939 00007f35c194d1e0 00007f35c1a160c4 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:27.902788 (+1138081us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16280320}
W20260709 13:15:27.945561 11518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.257s	user 0.010s	sys 0.000s
I20260709 13:15:27.950913 13428 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:27.951743 13596 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.940519 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:27.965694 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:27.982885 11370 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.013s	sys 0.006s
I20260709 13:15:27.989318 13537 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:27.993773 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 828481us
W20260709 13:15:27.996480 11483 leader_election.cc:336] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.663s (SENT)
W20260709 13:15:28.001442 11517 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1366) took 1859 ms (client timeout 1748 ms). Trace:
W20260709 13:15:28.007663 11408 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.748s (SENT) [suppressed 24 similar messages]
I20260709 13:15:28.008198 13534 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.008662 13138 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:28.009871 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.011308 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2023) took 1281 ms. Trace:
W20260709 13:15:28.032658 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.041487 11515 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.042341 13229 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:28.047202 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 461) took 4134 ms. Trace:
I20260709 13:15:28.047981 11442 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:28.050320 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.064900 11268 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39438 (request call id 358) took 13015 ms (client timeout 15000 ms). Trace:
I20260709 13:15:28.076058 13498 consensus_queue.cc:1059] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:28.092609 11364 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:27.584549 11445 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:27.787595 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.113695 13472 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:15:28.107501 11439 rpcz_store.cc:269] 0709 13:15:26.686844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.737228 (+ 50384us) service_pool.cc:224] Handling call
0709 13:15:28.106915 (+1369687us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:28.106417 11513 rpcz_store.cc:269] 0709 13:15:26.475154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.500408 (+ 25254us) service_pool.cc:224] Handling call
0709 13:15:28.105929 (+1605521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.116456 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:28.120513 13297 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.121873 13607 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:28.123926 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 458) took 1390 ms. Trace:
I20260709 13:15:28.127300 13476 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:28.129340 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.112979 11510 rpcz_store.cc:276] 0709 13:15:26.753422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.110846 (+1357424us) service_pool.cc:224] Handling call
0709 13:15:28.112543 (+  1697us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.130229 11262 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:28.128523 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 54427.8us
I20260709 13:15:28.147254 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2127) took 1539 ms. Trace:
I20260709 13:15:28.148769 13404 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:27.528625 13353 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:28.208323 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.238612 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 242394us
I20260709 13:15:28.239679 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:28.239064 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:28.240867 12066 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:28.173079 13336 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.239852 13314 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.241425 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:28.242508 13314 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.243847 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:28.244103 13440 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.244769 13440 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.245388 13314 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.245740 13515 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.237171 11438 rpcz_store.cc:276] 0709 13:15:26.608255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.166192 (+557937us) service_pool.cc:224] Handling call
0709 13:15:28.147182 (+980990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.163264 13141 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.247422 11361 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:28.247694 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 453) took 1543 ms. Trace:
I20260709 13:15:28.248126 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.248705 13544 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:28.249126 13544 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.249521 13544 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.249898 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.250325 13353 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:28.250891 11518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.563s	user 0.012s	sys 0.001s
I20260709 13:15:28.251994 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1704) took 4485 ms. Trace:
I20260709 13:15:28.250612 11438 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:28.248526 13515 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.253623 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.251873 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:28.254640 13515 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.254794 13609 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:28.256018 11262 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:28.258988 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:28.249110 13141 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.259965 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.260298 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:28.260954 11437 rpcz_store.cc:276] 0709 13:15:26.568139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.568247 (+   108us) service_pool.cc:224] Handling call
0709 13:15:26.568485 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:27.100988 (+532503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:27.868766 (+767778us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b540028a5d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:27.869090 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451544448}
I20260709 13:15:28.262689 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:28.263490 13472 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:28.264628 13472 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.265722 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:28.266834 11368 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:28.267174 13584 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:28.267897 11367 raft_consensus.cc:2368] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5.
I20260709 13:15:28.266552 11512 rpcz_store.cc:276] 0709 13:15:23.913144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.733723 (+2820579us) service_pool.cc:224] Handling call
0709 13:15:26.767390 (+ 33667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.046642 (+1279252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.046979 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.268340 13584 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.269268 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:28.269537 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.269783 11367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2018) took 1877 ms (client timeout 1960 ms). Trace:
W20260709 13:15:28.270469 11367 rpcz_store.cc:269] 0709 13:15:26.392512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.392628 (+   116us) service_pool.cc:224] Handling call
0709 13:15:28.269554 (+1876926us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54000c7454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.269732 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4217910528}
I20260709 13:15:28.271009 13138 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.271274 11512 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.271662 13138 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.271880 13534 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.271672 13563 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:28.266185 11363 rpcz_store.cc:276] 0709 13:15:26.729980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.732350 (+  2370us) service_pool.cc:224] Handling call
0709 13:15:26.732672 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:27.065232 (+332560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:27.065259 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:27.989271 (+924012us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b2c000c7fc8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:27.990076 (+   805us) log.cc:844] Serialized 26 byte log entry
0709 13:15:28.006132 (+ 16056us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:28.006195 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.006294 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:28.010678 (+  4384us) raft_consensus.cc:1685] finished
0709 13:15:28.010700 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.011051 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75642,"raft.queue_time_us":333453,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":14633,"spinlock_wait_cycles":11069184,"thread_start_us":15533,"threads_started":1,"wal-append.queue_time_us":15999}
W20260709 13:15:28.250418 11445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 369) took 1603 ms (client timeout 1840 ms). Trace:
I20260709 13:15:28.277406 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:28.277765 11445 rpcz_store.cc:269] 0709 13:15:26.647166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.649760 (+  2594us) service_pool.cc:224] Handling call
0709 13:15:27.583761 (+934001us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b5400314b28. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27db924 00007f35c19c4227 00007f35c19c3ab8 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6
0709 13:15:28.250361 (+666600us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22933248}
I20260709 13:15:28.279027 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.279758 11445 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 1.
I20260709 13:15:28.289048 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1852) took 1181 ms. Trace:
W20260709 13:15:28.280678 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.289404 11442 rpcz_store.cc:276] 0709 13:15:27.108166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.163932 (+ 55766us) service_pool.cc:224] Handling call
0709 13:15:28.288854 (+1124922us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b54003151d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.289000 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100252032}
I20260709 13:15:28.278203 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:28.290508 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.272344 12066 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:28.290917 11367 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:28.291680 13439 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.282541 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:28.282749 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.294070 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:28.294742 11439 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:28.296015 11516 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.610s	user 0.013s	sys 0.023s
I20260709 13:15:28.297978 13464 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:28.298432 13464 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.298504 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:28.299767 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.286621 11512 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
W20260709 13:15:28.316215 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.015s	sys 0.016s
I20260709 13:15:28.316737 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.317598 11445 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:28.318040 11512 raft_consensus.cc:1217] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Deduplicated request from leader. Original: 2.5->[4.6-4.6]   Dedup: 4.6->[]
I20260709 13:15:28.318704 11512 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:15:28.287535 11364 rpcz_store.cc:276] 0709 13:15:26.704269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.705055 (+   786us) service_pool.cc:224] Handling call
0709 13:15:26.705381 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.246249 (+1540868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.247352 (+  1103us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000c6554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:28.247636 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3189318400}
I20260709 13:15:28.266050 13472 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:28.270517 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:28.289896 13545 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.266985 13498 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:28.271341 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 155107us
I20260709 13:15:28.279239 11363 pending_rounds.cc:85] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 5
I20260709 13:15:28.295802 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:28.284600 11409 connection.cc:256] RPC call timeout handler was delayed by -1.6900160680002045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 13:15:28.301048 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:28.302703 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:28.302971 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:28.297506 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1696) took 4655 ms. Trace:
I20260709 13:15:28.304088 13594 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.305109 13432 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:28.305471 11370 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.014s	sys 0.007s
I20260709 13:15:28.306341 13576 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.308362 11438 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:28.357383 13545 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.358213 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 364) took 1787 ms. Trace:
I20260709 13:15:28.358728 11438 rpcz_store.cc:276] 0709 13:15:26.570869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.253281 (+1682412us) service_pool.cc:224] Handling call
0709 13:15:28.357920 (+104639us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b54002ea754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.358103 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123218688}
I20260709 13:15:28.360354 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.360879 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1690) took 4835 ms. Trace:
I20260709 13:15:28.361258 11438 raft_consensus.cc:2393] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:28.260365 13596 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.361308 11519 rpcz_store.cc:276] 0709 13:15:23.525665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.445916 (+2920251us) service_pool.cc:224] Handling call
0709 13:15:26.446279 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:26.484344 (+ 38065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:27.947079 (+1462735us) spinlock_profiling.cc:243] Waited 8.54 ms on lock 0x7b540043e7a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc975dc0 00007f35bca01a02 00007f35bc9fddd6 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:28.328830 (+381751us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540043e7a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc97b41d 00007f35bca01a57 00007f35bc9fddd6 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:28.360723 (+ 31893us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884893824}
I20260709 13:15:28.363617 13596 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.274724 13571 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:28.364796 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:28.365653 11519 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:28.366133 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2024) took 1635 ms. Trace:
I20260709 13:15:28.367312 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:28.366564 11370 rpcz_store.cc:276] 0709 13:15:26.730601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.733073 (+  2472us) service_pool.cc:224] Handling call
0709 13:15:26.734006 (+   933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:27.064259 (+330253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:27.064289 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:27.064742 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:15:27.813929 (+749187us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b4800002958. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f 00007f35c1a15f37 00007f35c1a2a55c 00007f35c1a2a508 00007f35c1a2a488 00007f35c1a2a443 00007f35c1a296ec 00007f35c0953a96
0709 13:15:28.365609 (+551680us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:28.365693 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.365793 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:28.365861 (+    68us) raft_consensus.cc:1685] finished
0709 13:15:28.365874 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.366075 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":329881,"raft.queue_time_us":332464,"raft.run_cpu_time_us":4971,"raft.run_wall_time_us":740752,"spinlock_wait_cycles":18017664,"thread_start_us":1281028,"threads_started":1,"wal-append.queue_time_us":522240}
I20260709 13:15:28.367996 11519 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.368690 11439 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
W20260709 13:15:28.370589 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 93306.7us
I20260709 13:15:28.311506 13229 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.312616 13533 raft_consensus.cc:3055] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:28.312915 11444 rpcz_store.cc:276] 0709 13:15:26.569867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.569973 (+   106us) service_pool.cc:224] Handling call
0709 13:15:26.570211 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.905817 (+335606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:26.905850 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:26.905894 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:26.905924 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:27.912770 (+1006846us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b18000e61a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419
0709 13:15:27.913548 (+   778us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400438fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:27.913598 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:27.913820 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":20982,"spinlock_wait_cycles":777712768}
I20260709 13:15:28.289803 13138 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.314796 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:28.268864 13297 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.190428 13478 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:28.386624 13478 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:28.388149 13478 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.260016 11369 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:28.321056 11515 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
W20260709 13:15:28.390709 11515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1369) took 2041 ms (client timeout 1918 ms). Trace:
W20260709 13:15:28.391077 11515 rpcz_store.cc:269] 0709 13:15:26.349807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.351969 (+  2162us) service_pool.cc:224] Handling call
0709 13:15:28.390504 (+2038535us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54002c0954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.390646 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3578897664}
I20260709 13:15:28.391677 13499 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:28.392190 13499 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:28.392575 13499 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.270511 13593 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:28.242132 12063 pending_rounds.cc:85] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 5
I20260709 13:15:28.322710 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:28.322844 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:28.281869 13607 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.291235 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1836) took 1658 ms. Trace:
W20260709 13:15:28.327111 11409 connection.cc:256] RPC call timeout handler was delayed by -1.6729160680001769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:15:28.319427 11517 rpcz_store.cc:269] 0709 13:15:26.141733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.227936 (+ 86203us) service_pool.cc:224] Handling call
0709 13:15:28.001056 (+1773120us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400109354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.001274 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007245696}
I20260709 13:15:28.265267 13581 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:28.326493 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 319000us
I20260709 13:15:28.207981 13316 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:28.333511 11367 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:28.334347 11513 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.334666 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:28.334959 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:28.335363 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.343901 12065 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:28.319492 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 462) took 4406 ms. Trace:
I20260709 13:15:28.363129 11366 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:28.310808 12033 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:28.309679 13553 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.269645 13584 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.375689 13533 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:28.247973 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2025) took 1181 ms. Trace:
I20260709 13:15:28.274130 13534 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.326586 11363 pending_rounds.cc:107] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:15:28.241446 11365 rpcz_store.cc:276] 0709 13:15:26.733790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.741946 (+  8156us) service_pool.cc:224] Handling call
0709 13:15:26.742297 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:28.107886 (+1365589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.123587 (+ 15701us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001e5fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:28.123869 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501522176}
W20260709 13:15:28.270578 11268 rpcz_store.cc:269] 0709 13:15:15.048954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.049359 (+   405us) service_pool.cc:224] Handling call
0709 13:15:16.546243 (+1496884us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b5400020710. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98da0b2 00007f35c2853a89 00007f35c27ebda6 00007f35c27bd088 00007f35c28bb37e 00007f35c0d7c2a4 00007f35c0d7c232 00007f35c0d7c161 00007f35c0d7c0dc
0709 13:15:16.925414 (+379171us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b5400020710. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9e9c 00007f35b98da0f6 00007f35c2853a89 00007f35c27ebda6 00007f35c27bd088 00007f35c28bb37e 00007f35c0d7c2a4 00007f35c0d7c232 00007f35c0d7c161 00007f35c0d7c0dc
0709 13:15:28.064591 (+11139177us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:25.788293 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:25.793962 (+  5669us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:25.793999 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:15:25.794019 (+    20us) tablet.cc:662] Decoding operations
0709 13:15:25.941596 (+147577us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:26.009405 (+ 67809us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:26.061141 (+ 51736us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:15:26.134238 (+ 73097us) tablet.cc:701] Row locks acquired
0709 13:15:26.137162 (+  2924us) write_op.cc:260] PREPARE: finished
0709 13:15:26.158070 (+ 20908us) write_op.cc:270] Start()
0709 13:15:26.171670 (+ 13600us) write_op.cc:276] Timestamp: P: 1783602926158003 usec, L: 0
0709 13:15:26.178443 (+  6773us) op_driver.cc:348] REPLICATION: starting
0709 13:15:26.287818 (+109375us) log.cc:844] Serialized 1289 byte log entry
0709 13:15:27.232331 (+944513us) op_driver.cc:464] REPLICATION: finished
0709 13:15:27.431031 (+198700us) write_op.cc:301] APPLY: starting
0709 13:15:27.431127 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:27.431339 (+   212us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:27.431970 (+   631us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:27.439135 (+  7165us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:27.439325 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:15:27.439375 (+    50us) write_op.cc:312] APPLY: finished
0709 13:15:27.439613 (+   238us) log.cc:844] Serialized 35 byte log entry
0709 13:15:27.440417 (+   804us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:27.440516 (+    99us) write_op.cc:454] Released schema lock
0709 13:15:27.440554 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:27.440881 (+   327us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:27.440907 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"spinlock_wait_cycles":134549632,"child_traces":[["op",{"apply.queue_time_us":182057,"apply.run_cpu_time_us":2263,"apply.run_wall_time_us":9838,"num_ops":3,"prepare.queue_time_us":372174,"prepare.run_cpu_time_us":41699,"prepare.run_wall_time_us":779996,"raft.queue_time_us":476060,"raft.run_cpu_time_us":14339,"raft.run_wall_time_us":261919,"replication_time_us":1003978,"spinlock_wait_cycles":1815424,"thread_start_us":988184,"threads_started":4,"wal-append.queue_time_us":339926}]]}
I20260709 13:15:28.329859 11516 rpcz_store.cc:276] 0709 13:15:23.642245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.577943 (+2935698us) service_pool.cc:224] Handling call
0709 13:15:26.592509 (+ 14566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.595544 (+  3035us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:26.685290 (+ 89746us) log.cc:844] Serialized 14 byte log entry
0709 13:15:28.296497 (+1611207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:28.296600 (+   103us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:28.296703 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.297305 (+   602us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b54002c1d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:28.297340 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.297455 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":96062,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":26271,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":6442368,"thread_start_us":1688166,"threads_started":2,"wal-append.queue_time_us":900031}
I20260709 13:15:28.252403 11518 rpcz_store.cc:276] 0709 13:15:23.766691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.650394 (+2883703us) service_pool.cc:224] Handling call
0709 13:15:26.650671 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:26.650856 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:26.687970 (+ 37114us) log.cc:844] Serialized 14 byte log entry
0709 13:15:28.251488 (+1563518us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:28.251610 (+   122us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:28.251695 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.251773 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.251941 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88811,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":1478,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":1599857,"threads_started":2,"wal-append.queue_time_us":753516}
I20260709 13:15:28.381616 13545 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.383293 13138 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.383778 12069 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:28.384400 11510 raft_consensus.cc:2410] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:28.385614 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.270807 13537 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.154723 13454 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.405966 11519 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:28.417050 11442 rpcz_store.cc:276] 0709 13:15:26.633078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.290224 (+1657146us) service_pool.cc:224] Handling call
0709 13:15:28.291165 (+   941us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:28.399798 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 123452us
I20260709 13:15:28.418851 12066 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.365175 13571 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.188040 13348 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.752139 12069 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:28.752888 11441 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:28.753516 12069 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.378911 13353 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.432215 13593 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:28.423753 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.653316 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.420696 13297 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.757406 11510 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:28.757251 13297 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.758315 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1708) took 4841 ms. Trace:
I20260709 13:15:28.416774 12063 pending_rounds.cc:107] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:15:28.260406 13544 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:28.760843 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 450162us
W20260709 13:15:28.761650 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.762539 12033 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:28.762776 11334 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:28.766808 13544 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.430711 13609 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.768232 13609 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:28.748561 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.596517 11515 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:28.432760 12060 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:28.763594 11438 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:28.484251 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:28.420389 11409 leader_election.cc:336] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.576s (SENT)
I20260709 13:15:28.427218 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:28.774819 11363 raft_consensus.cc:2889] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:28.764142 11441 raft_consensus.cc:2410] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:15:28.765941 11440 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:28.626242 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:28.843886 13440 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:28.844262 11439 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:28.844688 11511 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:28.844897 12069 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.844718 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:28.759884 12063 raft_consensus.cc:2889] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:28.425086 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.848495 11437 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:28.707623 11436 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:28.853629 11514 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
W20260709 13:15:28.854411 11514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1731) took 2115 ms (client timeout 2238 ms). Trace:
I20260709 13:15:28.855973 11519 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:28.856316 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.856640 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1712) took 3272 ms. Trace:
W20260709 13:15:28.857091 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.857049 11519 rpcz_store.cc:276] 0709 13:15:25.584438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.821652 (+3237214us) service_pool.cc:224] Handling call
0709 13:15:28.855779 (+ 34127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.856370 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.856563 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:28.856848 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.773218 13545 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:28.422796 13498 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.771097 11518 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:28.758801 11510 rpcz_store.cc:276] 0709 13:15:23.916932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.756449 (+4839517us) service_pool.cc:224] Handling call
0709 13:15:28.757043 (+   594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.757996 (+   953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.758219 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.755646 13593 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.815317 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 463) took 4861 ms. Trace:
I20260709 13:15:28.873703 11515 rpcz_store.cc:276] 0709 13:15:23.914475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.587243 (+4672768us) service_pool.cc:224] Handling call
0709 13:15:28.596299 (+  9056us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.775433 (+179134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.775616 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.419857 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:28.774067 11445 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.755059 13353 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.765933 13348 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.879091 11438 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:28.879905 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 365) took 2308 ms. Trace:
I20260709 13:15:28.857825 13404 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.881201 13353 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:28.881423 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 119020us
W20260709 13:15:28.882752 11436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2122) took 2312 ms (client timeout 1982 ms). Trace:
W20260709 13:15:28.883188 11436 rpcz_store.cc:269] 0709 13:15:26.570141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.667916 (+ 97775us) service_pool.cc:224] Handling call
0709 13:15:28.882497 (+2214581us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54003156d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.882659 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3742104832}
I20260709 13:15:28.885105 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 366) took 2313 ms. Trace:
I20260709 13:15:28.885475 11436 rpcz_store.cc:276] 0709 13:15:26.571352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.884159 (+2312807us) service_pool.cc:224] Handling call
0709 13:15:28.884548 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.884762 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.884789 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:28.884848 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:28.884878 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.884930 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.885028 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.690683 13476 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.843017 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:28.888055 11440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1851) took 1786 ms (client timeout 2061 ms). Trace:
I20260709 13:15:28.888469 11444 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:28.889156 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2126) took 2283 ms. Trace:
W20260709 13:15:28.889304 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 330321us
I20260709 13:15:28.889484 11444 rpcz_store.cc:276] 0709 13:15:26.605360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.887974 (+2282614us) service_pool.cc:224] Handling call
0709 13:15:28.888302 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.888879 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.889085 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.890219 11335 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:28.890745 11444 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:15:28.890910 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1692) took 5336 ms. Trace:
I20260709 13:15:28.763849 11361 rpcz_store.cc:276] 0709 13:15:27.066724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.072387 (+  5663us) service_pool.cc:224] Handling call
0709 13:15:28.247897 (+1175510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.815593 13478 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.839581 13138 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:28.871285 13545 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.431002 13421 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.726949 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.764802 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:28.887180 13476 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.767465 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:28.881125 11517 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.863541 12067 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.755746 13454 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:28.873281 13316 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:28.861171 12090 debug-util.cc:398] Leaking SignalData structure 0x7b08000e9020 after lost signal to thread 10260
I20260709 13:15:28.751441 13444 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:28.842212 13594 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.856768 11513 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:15:28.516731 11483 leader_election.cc:336] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.811s (SENT)
I20260709 13:15:28.824453 13581 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:28.777263 13596 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:15:28.887295 11436 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:28.875378 12069 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:15:28.878167 13426 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.892621 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 595) took 4515 ms. Trace:
I20260709 13:15:28.878775 11519 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:28.953737 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1714) took 3368 ms. Trace:
I20260709 13:15:28.954087 11519 rpcz_store.cc:276] 0709 13:15:25.584899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.870816 (+3285917us) service_pool.cc:224] Handling call
0709 13:15:28.878370 (+  7554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.953519 (+ 75149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.953686 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.839207 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:28.880580 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1718) took 3140 ms. Trace:
I20260709 13:15:28.843027 13229 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:28.891361 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1835) took 2266 ms. Trace:
I20260709 13:15:28.897418 11442 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:28.898242 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 130919us
I20260709 13:15:28.949348 12069 pending_rounds.cc:85] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 5
I20260709 13:15:28.597018 13464 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:15:28.897725 13439 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.606s	user 0.007s	sys 0.000s
W20260709 13:15:28.854919 11514 rpcz_store.cc:269] 0709 13:15:26.738540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.149727 (+1411187us) service_pool.cc:224] Handling call
0709 13:15:28.854313 (+704586us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:28.960624 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 183281us
I20260709 13:15:28.891359 11511 rpcz_store.cc:276] 0709 13:15:23.554180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.469947 (+2915767us) service_pool.cc:224] Handling call
0709 13:15:28.780347 (+2310400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.845172 (+ 64825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.890691 (+ 45519us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54002c0950. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.890822 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4360511232}
I20260709 13:15:28.905593 11443 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:28.767289 11512 rpcz_store.cc:276] 0709 13:15:23.913384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.317188 (+4403804us) service_pool.cc:224] Handling call
0709 13:15:28.317818 (+   630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:28.319155 (+  1337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.319394 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.872570 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 465) took 4208 ms. Trace:
I20260709 13:15:28.842978 13297 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:28.929988 11441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1854) took 1749 ms (client timeout 1727 ms). Trace:
W20260709 13:15:28.964596 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 546760us
W20260709 13:15:28.949697 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 80546us
I20260709 13:15:28.764135 13533 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.956003 12069 pending_rounds.cc:107] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:15:28.965386 12069 raft_consensus.cc:2889] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:28.964061 11518 rpcz_store.cc:276] 0709 13:15:24.664527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.770567 (+4106040us) service_pool.cc:224] Handling call
0709 13:15:28.770939 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.872288 (+101349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.872502 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.927829 13444 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:28.956480 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2123) took 2376 ms. Trace:
I20260709 13:15:28.968910 11436 rpcz_store.cc:276] 0709 13:15:26.573998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.886483 (+2312485us) service_pool.cc:224] Handling call
0709 13:15:28.886885 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.950134 (+ 63249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.950328 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.898916 13476 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:28.888473 11440 rpcz_store.cc:269] 0709 13:15:27.101884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.128039 (+1026155us) service_pool.cc:224] Handling call
0709 13:15:28.887848 (+759809us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400398d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.887983 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136597888}
I20260709 13:15:28.955330 11515 rpcz_store.cc:276] 0709 13:15:25.740596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.879649 (+3139053us) service_pool.cc:224] Handling call
0709 13:15:28.879985 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.880269 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.880293 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:28.880333 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:28.880369 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.880417 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.880509 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.895817 13545 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:28.908385 12090 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc240 after lost signal to thread 11302
I20260709 13:15:28.956015 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:28.880270 11438 rpcz_store.cc:276] 0709 13:15:26.571106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.875728 (+2304622us) service_pool.cc:224] Handling call
0709 13:15:28.878917 (+  3189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.879617 (+   700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.879829 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.928763 11445 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:28.978413 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1855) took 1001 ms. Trace:
I20260709 13:15:28.898460 13440 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:28.895978 13138 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:28.964293 11441 rpcz_store.cc:269] 0709 13:15:27.180645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.940091 (+759446us) service_pool.cc:224] Handling call
0709 13:15:28.929766 (+989675us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b5400278dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:28.929909 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118574976}
I20260709 13:15:28.929270 11517 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:28.950071 13482 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.955479 11444 rpcz_store.cc:276] 0709 13:15:26.625176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.890219 (+2265043us) service_pool.cc:224] Handling call
0709 13:15:28.890556 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:28.891150 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.891319 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.956856 11516 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:28.906482 13537 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.896111 13421 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.905479 13316 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.978825 11445 rpcz_store.cc:276] 0709 13:15:27.977445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.325667 (+348222us) service_pool.cc:224] Handling call
0709 13:15:28.978356 (+652689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.881126 13404 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:28.964901 13348 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.982118 12034 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:28.910668 13498 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.949632 13426 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.988148 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1355) took 4208 ms. Trace:
I20260709 13:15:28.962345 13439 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:28.975734 13553 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:28.992183 11518 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:28.996202 11516 rpcz_store.cc:276] 0709 13:15:24.780013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.771370 (+3991357us) service_pool.cc:224] Handling call
0709 13:15:28.930037 (+158667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.987974 (+ 57937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.988102 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:28.982817 13297 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:28.984411 13138 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.025442 11511 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:29.033898 12090 debug-util.cc:398] Leaking SignalData structure 0x7b0800265440 after lost signal to thread 11394
W20260709 13:15:29.034260 12090 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0800 after lost signal to thread 11395
W20260709 13:15:29.035202 12090 debug-util.cc:398] Leaking SignalData structure 0x7b08002c7be0 after lost signal to thread 11469
W20260709 13:15:29.036171 12090 debug-util.cc:398] Leaking SignalData structure 0x7b0800297ac0 after lost signal to thread 11543
I20260709 13:15:28.989955 13609 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:28.971244 13584 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:15:29.051309 13426 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:28.952163 12063 rpcz_store.cc:276] 0709 13:15:24.377323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.652653 (+275330us) service_pool.cc:224] Handling call
0709 13:15:24.704498 (+ 51845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.848052 (+4143554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.892272 (+ 44220us) spinlock_profiling.cc:243] Waited 2.44 s on lock 0x7b54004b7154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:28.892540 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5612233088}
I20260709 13:15:29.037339 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1727) took 2544 ms. Trace:
I20260709 13:15:29.052606 11518 rpcz_store.cc:276] 0709 13:15:26.475332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.991678 (+2516346us) service_pool.cc:224] Handling call
0709 13:15:28.992013 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.019628 (+ 27615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.019830 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.054178 11518 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:29.054741 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1375) took 2301 ms. Trace:
W20260709 13:15:29.041961 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63493.6us
W20260709 13:15:29.042665 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 152620us
W20260709 13:15:29.018038 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.055109 11518 rpcz_store.cc:276] 0709 13:15:26.753422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.053436 (+2300014us) service_pool.cc:224] Handling call
0709 13:15:29.053809 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.054525 (+   716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.054689 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.044786 11513 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:29.045053 12090 debug-util.cc:398] Leaking SignalData structure 0x7b08003b01e0 after lost signal to thread 12093
W20260709 13:15:29.047705 13499 consensus_meta.cc:287] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.655s	user 0.003s	sys 0.007s
I20260709 13:15:29.049233 13421 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.044320 13617 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:29.044427 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 624097us
I20260709 13:15:29.056945 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1365) took 2896 ms. Trace:
I20260709 13:15:29.058249 11443 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:29.054629 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:29.061084 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 544364us
I20260709 13:15:29.037263 13138 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:29.103195 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 219926us
I20260709 13:15:29.103652 11444 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:29.071449 12090 debug-util.cc:398] Leaking SignalData structure 0x7b0800375800 after lost signal to thread 12094
I20260709 13:15:29.111346 11440 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:15:29.094386 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.104913 11511 rpcz_store.cc:276] 0709 13:15:26.141475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.980579 (+2839104us) service_pool.cc:224] Handling call
0709 13:15:29.025243 (+ 44664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.037554 (+ 12311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.037762 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:29.107074 11333 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.002s	sys 0.021s
I20260709 13:15:29.054170 13593 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:29.107336 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1838) took 2465 ms. Trace:
W20260709 13:15:29.109438 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.024400 13348 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.117527 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:29.113341 11333 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.002s	sys 0.021s
W20260709 13:15:29.107376 13472 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.782s	user 0.007s	sys 0.002s
I20260709 13:15:28.989704 13482 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:29.127238 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1839) took 2444 ms (client timeout 2106 ms). Trace:
W20260709 13:15:29.127663 11443 rpcz_store.cc:269] 0709 13:15:26.682443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.916891 (+1234448us) service_pool.cc:224] Handling call
0709 13:15:29.127073 (+1210182us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b540027a1d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:29.127178 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526872448}
I20260709 13:15:29.128412 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2121) took 2553 ms. Trace:
I20260709 13:15:29.109961 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1367) took 2960 ms. Trace:
I20260709 13:15:29.105708 13617 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.119458 11442 rpcz_store.cc:276] 0709 13:15:26.637346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.974659 (+2337313us) service_pool.cc:224] Handling call
0709 13:15:28.996411 (+ 21752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.055161 (+ 58750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.055188 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.102602 (+ 47414us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:29.102646 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.102712 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.102843 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.121824 13440 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:29.104079 11436 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:29.122633 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:29.122385 11483 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.958s (SENT)
W20260709 13:15:29.124693 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 749556us
I20260709 13:15:29.129604 11437 rpcz_store.cc:276] 0709 13:15:26.569894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.849199 (+2279305us) service_pool.cc:224] Handling call
0709 13:15:28.849560 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.849857 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:28.956803 (+106946us) log.cc:844] Serialized 14 byte log entry
0709 13:15:29.122695 (+165892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.122745 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:29.122791 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.122850 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.122968 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":139307,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":12997,"thread_start_us":239164,"threads_started":2}
I20260709 13:15:29.115722 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1844) took 2377 ms. Trace:
I20260709 13:15:29.114930 13593 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.132620 13440 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:29.128582 13464 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:29.059593 13444 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.101948 13499 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.115463 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1364) took 2972 ms. Trace:
I20260709 13:15:29.135569 13617 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:29.141322 11516 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:29.152458 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1374) took 2517 ms. Trace:
I20260709 13:15:29.141825 11511 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:29.152706 12067 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:29.139561 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:29.148700 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 111511us
W20260709 13:15:29.144146 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.149523 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.154102 11516 rpcz_store.cc:276] 0709 13:15:26.634909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.004473 (+2369564us) service_pool.cc:224] Handling call
0709 13:15:29.140883 (+136410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.152213 (+ 11330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.152406 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.140585 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1848) took 2066 ms. Trace:
I20260709 13:15:29.149714 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 368) took 2486 ms. Trace:
I20260709 13:15:29.154806 11436 rpcz_store.cc:276] 0709 13:15:26.646896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.983736 (+2336840us) service_pool.cc:224] Handling call
0709 13:15:29.037229 (+ 53493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.133048 (+ 95819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.133289 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.154707 11443 rpcz_store.cc:276] 0709 13:15:27.074432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.131794 (+2057362us) service_pool.cc:224] Handling call
0709 13:15:29.139973 (+  8179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.140221 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.140249 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.140311 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:29.140338 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.140389 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.140500 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.115983 13316 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:29.130290 11517 rpcz_store.cc:276] 0709 13:15:26.150174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.984328 (+2834154us) service_pool.cc:224] Handling call
0709 13:15:28.997050 (+ 12722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.109581 (+112531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.109615 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.109660 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:29.109688 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.109775 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.109901 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12084,"raft.queue_time_us":3707,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":3647}
I20260709 13:15:29.153106 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1733) took 1053 ms. Trace:
I20260709 13:15:29.160409 11511 rpcz_store.cc:276] 0709 13:15:28.099430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.116590 (+1017160us) service_pool.cc:224] Handling call
0709 13:15:29.141472 (+ 24882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.152884 (+ 11412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.153055 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.136727 13440 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.162647 13454 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:29.162523 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 46395us
I20260709 13:15:29.151758 11513 rpcz_store.cc:276] 0709 13:15:26.141204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.973904 (+2832700us) service_pool.cc:224] Handling call
0709 13:15:29.044592 (+ 70688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.102275 (+ 57683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.113133 (+ 10858us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.164016 13348 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.164438 11445 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:15:29.162810 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 45891.3us
W20260709 13:15:29.142099 13616 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:15:29.165759 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1376) took 2389 ms. Trace:
I20260709 13:15:29.168852 11518 rpcz_store.cc:276] 0709 13:15:26.776639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.056121 (+2279482us) service_pool.cc:224] Handling call
0709 13:15:29.145374 (+ 89253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.165464 (+ 20090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.165496 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.165534 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:29.165564 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.165619 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.165712 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.165701 11333 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.165064 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1847) took 2277 ms. Trace:
I20260709 13:15:29.170300 11445 rpcz_store.cc:276] 0709 13:15:26.887897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.114440 (+2226543us) service_pool.cc:224] Handling call
0709 13:15:29.151910 (+ 37470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.164801 (+ 12891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.165013 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.139528 13444 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:29.162850 13426 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:29.177354 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:29.178416 13439 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:29.179687 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 70971.9us
I20260709 13:15:29.179459 13593 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.180480 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.180770 13593 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.181828 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.183090 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.167735 13472 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:29.169072 13454 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.191304 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.191464 13626 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:29.190966 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:29.195088 11366 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:29.193815 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.183761 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1846) took 2430 ms. Trace:
I20260709 13:15:29.195822 11442 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:29.204526 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.195941 11440 rpcz_store.cc:276] 0709 13:15:26.753725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.999530 (+2245805us) service_pool.cc:224] Handling call
0709 13:15:29.109306 (+109776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.111827 (+  2521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.175894 (+ 64067us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b54002094d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:29.183703 (+  7809us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6086528}
I20260709 13:15:29.205745 11445 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:29.214732 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1853) took 2087 ms. Trace:
I20260709 13:15:29.215355 11442 rpcz_store.cc:276] 0709 13:15:27.127060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.159954 (+2032894us) service_pool.cc:224] Handling call
0709 13:15:29.170193 (+ 10239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.214472 (+ 44279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.214675 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.220248 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1843) took 2495 ms. Trace:
I20260709 13:15:29.183017 13499 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.223242 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2130) took 1224 ms. Trace:
I20260709 13:15:29.171305 13229 consensus_queue.cc:1059] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:15:29.140910 11438 rpcz_store.cc:276] 0709 13:15:26.735936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.019417 (+2283481us) service_pool.cc:224] Handling call
0709 13:15:29.063099 (+ 43682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.063303 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.063338 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.063380 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:29.066499 (+  3119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.107723 (+ 41224us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.113428 (+  5705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.226178 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1363) took 3081 ms. Trace:
I20260709 13:15:29.220705 11444 rpcz_store.cc:276] 0709 13:15:26.725097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.996122 (+2271025us) service_pool.cc:224] Handling call
0709 13:15:29.038145 (+ 42023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.107960 (+ 69815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.206622 (+ 98662us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b540028a0d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:29.220175 (+ 13553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60018432}
I20260709 13:15:29.172719 13316 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.225344 11445 rpcz_store.cc:276] 0709 13:15:27.990773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.187273 (+1196500us) service_pool.cc:224] Handling call
0709 13:15:29.205550 (+ 18277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.215233 (+  9683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.215447 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.181831 11439 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:29.230635 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:29.192142 11361 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:29.240352 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:29.237519 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.220723 13594 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:29.222050 13576 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:29.238404 11437 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:15:29.241396 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.244976 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 90892us
I20260709 13:15:29.230784 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:29.245244 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.191666 13454 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.188243 13571 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:15:29.243861 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:29.246992 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.248991 11369 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:29.246711 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.228832 13421 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:29.231437 13439 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.251304 11367 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.248426 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:29.222545 13620 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:15:29.256122 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 73117.5us
I20260709 13:15:29.263859 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.264346 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.264873 12067 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:29.264950 12069 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:29.266626 12066 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:29.248278 13626 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:29.246690 13464 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:29.289806 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 61378.3us
I20260709 13:15:29.251112 13594 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.267735 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.226517 13426 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:15:29.298611 13624 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:29.299163 13623 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:15:29.292713 13464 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.300223 13464 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.291003 13404 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.298410 13426 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:29.294775 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.292542 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:29.264467 11370 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:29.290692 13476 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.246472 13499 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:15:29.267930 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.303001 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.298523 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.303506 11366 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:29.289367 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 371) took 1039 ms. Trace:
I20260709 13:15:29.295193 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.291383 13626 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.313428 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:29.313201 11443 rpcz_store.cc:276] 0709 13:15:28.250059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.221206 (+971147us) service_pool.cc:224] Handling call
0709 13:15:29.229891 (+  8685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.262305 (+ 32414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.262332 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.288467 (+ 26135us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:29.288513 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.289202 (+   689us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.289314 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.293999 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.289691 13498 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:29.304113 13426 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.298827 13472 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:29.306013 13476 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.319761 13545 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:29.320974 13627 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:15:29.321400 13637 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:29.307150 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:29.323624 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:29.324406 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:29.325261 13639 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:29.325984 11368 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:29.326496 13631 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:29.326647 11443 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:29.326997 13464 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.338897 13472 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.294291 13571 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:15:29.389241 13472 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:29.337118 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.338277 12067 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:29.339103 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.390272 11366 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:29.302348 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2129) took 1312 ms. Trace:
I20260709 13:15:29.391526 11436 rpcz_store.cc:276] 0709 13:15:27.990430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.172397 (+1181967us) service_pool.cc:224] Handling call
0709 13:15:29.192604 (+ 20207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.192795 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:29.230680 (+ 37885us) log.cc:844] Serialized 14 byte log entry
0709 13:15:29.301902 (+ 71222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.301969 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:29.302026 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.302104 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.302272 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38410,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1324,"raft.queue_time_us":144,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":9885,"thread_start_us":50513,"threads_started":1,"wal-append.queue_time_us":128358}
I20260709 13:15:29.339993 11367 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:29.341491 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.346949 13426 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:29.347647 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 58730.1us
I20260709 13:15:29.348577 11370 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:29.349203 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1737) took 1233 ms. Trace:
I20260709 13:15:29.350067 13316 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.344875 13476 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.315977 13626 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.351784 13594 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.353361 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.403733 12065 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:29.353395 11442 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:29.354039 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.354215 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.355115 13619 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:29.355233 12068 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 3 LEARNER]: Advancing to term 5
I20260709 13:15:29.355715 13478 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:29.355962 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.353626 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.325301 11362 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:15:29.408213 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54268.2us
W20260709 13:15:29.345976 11409 leader_election.cc:336] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.986s (SENT)
I20260709 13:15:29.353633 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1841) took 2664 ms. Trace:
W20260709 13:15:29.360157 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.356525 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:29.356030 13615 log.cc:927] Time spent T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.385s	user 0.003s	sys 0.000s
I20260709 13:15:29.286970 13439 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.365775 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.365990 11443 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:29.366760 13545 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:29.365710 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.367692 13464 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:29.357640 12063 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:29.233085 11519 rpcz_store.cc:276] 0709 13:15:26.140946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.960439 (+2819493us) service_pool.cc:224] Handling call
0709 13:15:28.980383 (+ 19944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.996337 (+ 15954us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:29.063570 (+ 67233us) log.cc:844] Serialized 14 byte log entry
0709 13:15:29.205128 (+141558us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.205207 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:29.221722 (+ 16515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.221800 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.221927 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26885,"prepare.queue_time_us":44198,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":48916,"thread_start_us":112819,"threads_started":1,"wal-append.queue_time_us":166332}
I20260709 13:15:29.370616 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:29.371215 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 47696.8us
I20260709 13:15:29.372319 13138 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.372898 13353 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:29.375455 13533 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.376662 11369 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:29.407117 13581 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:29.380052 13634 raft_consensus.cc:3055] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:29.380458 13636 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:29.455627 13545 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.456195 13545 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.379240 13482 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:29.383946 11439 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:29.361357 13593 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:15:29.388132 11365 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:29.466424 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.361284 13639 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.445992 13454 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:29.468950 13639 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.470602 13639 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:29.362800 13596 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:15:29.427678 13631 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:15:29.356881 13637 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.476574 13439 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:29.394033 11437 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:29.395776 11515 rpcz_store.cc:276] 0709 13:15:28.115927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.116810 (+1000883us) service_pool.cc:224] Handling call
0709 13:15:29.152261 (+ 35451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:29.182227 (+ 29966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.182256 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:29.182771 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:15:29.190588 (+  7817us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:29.190629 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.190704 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:29.349005 (+158301us) raft_consensus.cc:1685] finished
0709 13:15:29.349023 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.349152 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26848,"raft.queue_time_us":9761,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":5246,"thread_start_us":7439,"threads_started":1,"wal-append.queue_time_us":57661}
I20260709 13:15:29.401415 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:29.402518 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 61113.7us
W20260709 13:15:29.404529 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.404544 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 372) took 1024 ms. Trace:
W20260709 13:15:29.406148 11483 leader_election.cc:336] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 1.960s (SENT)
I20260709 13:15:29.479040 13439 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:29.406757 13630 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
I20260709 13:15:29.479493 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.479535 13647 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:29.479598 13439 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:29.480460 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 76408.6us
I20260709 13:15:29.481348 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:29.482064 13464 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:29.402410 13623 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:29.395190 13641 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.416591 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1861) took 1022 ms. Trace:
I20260709 13:15:29.486984 11443 rpcz_store.cc:276] 0709 13:15:28.394276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.365564 (+971288us) service_pool.cc:224] Handling call
0709 13:15:29.365834 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.416335 (+ 50501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.416531 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.423784 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:29.364756 13421 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:29.428377 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.408849 13571 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.421635 13620 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:29.449478 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.496888 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.443675 13353 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.447314 13499 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:29.443482 12060 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:29.459720 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.408236 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.460742 11364 raft_consensus.cc:2393] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:29.461941 13426 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:29.464026 13642 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:15:29.465354 13229 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:29.415928 13584 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:29.455160 13634 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.409874 11441 rpcz_store.cc:276] 0709 13:15:26.689698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.983815 (+2294117us) service_pool.cc:224] Handling call
0709 13:15:28.989609 (+  5794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:28.991951 (+  2342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.991982 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:28.992461 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:15:29.193706 (+201245us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:29.193743 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.193801 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:29.353446 (+159645us) raft_consensus.cc:1685] finished
0709 13:15:29.353463 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.353584 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2391,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":61514,"thread_start_us":199668,"threads_started":1,"wal-append.queue_time_us":293878}
I20260709 13:15:29.478717 11442 rpcz_store.cc:276] 0709 13:15:28.380597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.352808 (+972211us) service_pool.cc:224] Handling call
0709 13:15:29.353157 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:29.404324 (+ 51167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.404502 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:29.493049 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 70393.5us
I20260709 13:15:29.455533 13636 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:29.454419 13472 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.494022 13439 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:29.499044 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.500190 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 129654us
I20260709 13:15:29.498559 13482 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:29.503618 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 102281us
I20260709 13:15:29.486769 13623 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.447086 13619 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.468122 12063 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:29.460659 13617 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.503348 13229 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.508160 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.508621 13643 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
W20260709 13:15:29.516424 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.501145 13316 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:29.535687 11335 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:29.532187 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.540145 13648 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:15:29.564605 12060 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:29.538905 13619 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.565615 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:29.535230 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1537) took 1256 ms. Trace:
W20260709 13:15:29.585669 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.534298 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.598816 12067 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:29.566430 11363 rpcz_store.cc:276] 0709 13:15:28.278504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.278659 (+   155us) service_pool.cc:224] Handling call
0709 13:15:28.278988 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:29.105570 (+826582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.105602 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:29.108836 (+  3234us) log.cc:844] Serialized 52 byte log entry
0709 13:15:29.361983 (+253147us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:29.362023 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.362376 (+   353us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:29.535008 (+172632us) raft_consensus.cc:1685] finished
0709 13:15:29.535028 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.535166 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94567,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4721,"thread_start_us":423354,"threads_started":2,"wal-append.queue_time_us":276282}
I20260709 13:15:29.536268 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:29.600018 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 260091us
I20260709 13:15:29.505125 13596 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:15:29.601079 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.601070 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 63088.6us
I20260709 13:15:29.502700 13499 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.623698 13499 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.611590 13646 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:29.501088 13571 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.625049 11440 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:29.624961 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:29.625785 11512 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 2.
I20260709 13:15:29.626384 13571 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.626992 12065 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:29.627254 11519 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:29.628727 11515 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:29.630297 13607 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:29.630921 13617 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:29.540431 13626 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.631699 13421 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:29.632293 13421 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:29.505048 12061 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:29.529557 13472 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:29.577445 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 98967.9us
I20260709 13:15:29.657648 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:29.583617 12068 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:29.540903 12062 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:29.658381 11440 raft_consensus.cc:2376] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4. Re-sending same reply.
I20260709 13:15:29.539927 13478 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:15:29.537187 13229 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.509117 13620 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.533226 13482 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:29.611572 11361 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:29.539587 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.612548 13594 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:29.613559 12069 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:29.537315 13545 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.508919 13650 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:29.660281 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:29.668231 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.668944 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 175212us
I20260709 13:15:29.660728 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:29.669337 12034 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.563216 13581 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:29.536351 13138 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:29.671262 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 135691us
W20260709 13:15:29.671656 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.673506 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:29.501319 13641 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.675170 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.624454 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:29.659267 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:29.618768 13596 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.678156 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:29.686655 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 191548us
I20260709 13:15:29.678613 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.679883 12067 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.678550 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:29.631775 13404 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:15:29.651314 13646 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.510515 13640 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.167s
I20260709 13:15:29.693915 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.694566 12065 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:29.669718 13621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.021s	sys 0.007s
I20260709 13:15:29.697100 11444 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:15:29.683851 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.683959 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:29.688103 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:29.688979 11515 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:29.689244 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.690654 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.691366 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:29.521935 13584 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.672530 13454 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.686538 11445 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:29.945639 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:29.683220 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:29.697120 12066 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.698719 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:29.563665 13654 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:15:29.699725 11334 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.701102 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:29.947870 12069 raft_consensus.cc:2410] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:29.948199 12068 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:29.701879 11436 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.670622 13348 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:15:29.702497 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:29.950354 11517 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:29.772680 13626 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.699241 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:29.810953 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 122919us
W20260709 13:15:29.709862 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:29.870635 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.537696 13353 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.797598 13649 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 13:15:29.929280 13650 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.689401 13593 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:29.801429 13581 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.535671 13634 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.535161 13636 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.671736 13476 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.699810 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:29.807060 13482 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:29.798574 13662 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:29.702175 13621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.021s	sys 0.007s
I20260709 13:15:29.803596 13620 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:29.931506 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.841521 13464 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.631497 13498 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:15:29.675849 12063 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:30.075609 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.077117 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:29.938719 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.691439 11513 raft_consensus.cc:2376] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3. Re-sending same reply.
I20260709 13:15:29.686826 13478 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:29.939554 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 241923us
I20260709 13:15:29.930365 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:30.081746 12065 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.930653 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.081862 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.023361 13639 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:29.527662 13623 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:29.951020 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 251365us
I20260709 13:15:29.929387 13647 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.083279 13348 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:29.938966 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:29.962244 12064 raft_consensus.cc:2410] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:29.963163 12062 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:29.963482 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:29.963632 13584 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:29.963982 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.954444 13229 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.949143 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:29.944775 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:29.960911 13619 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:15:29.958535 13657 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:29.967139 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:29.949828 13454 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:29.967324 13594 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:15:30.122673 13661 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.123117 13594 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:30.123596 13594 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.973925 13353 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.123806 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.124643 11514 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
I20260709 13:15:29.981746 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.125711 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.126981 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:29.986758 13297 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.128628 13297 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:30.130775 13297 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:29.989020 12066 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:29.992483 11436 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:30.133121 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:30.133683 11439 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:30.133864 11436 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:15:29.987821 11483 leader_election.cc:336] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.242s (SENT)
I20260709 13:15:30.134758 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:29.966523 13650 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.045653 11408 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:30.048161 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.055258 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.055567 13654 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:30.057593 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.094393 13464 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:30.067566 13662 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:30.068358 13644 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:30.069947 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.081436 13478 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:29.945300 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:30.083981 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:29.942233 12067 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:30.082983 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.082371 11516 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
W20260709 13:15:30.149130 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 65221.6us
W20260709 13:15:30.087319 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.968880 13593 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:15:30.090646 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 395588us
I20260709 13:15:30.083202 13639 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:30.083593 13647 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.115370 13657 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.087927 13348 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.121441 11515 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:30.111855 11510 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:29.972004 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:30.114972 11365 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:29.984647 13641 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:30.137744 13607 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:29.964764 13596 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:30.163053 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.135015 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:30.155891 13421 consensus_meta.cc:287] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.003s
I20260709 13:15:30.164253 11363 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:30.158272 13657 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:30.137300 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:30.165859 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:30.135808 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 170612us
W20260709 13:15:30.167800 11483 leader_election.cc:336] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.662s (SENT)
I20260709 13:15:30.170691 13626 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:30.136116 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 90557.4us
I20260709 13:15:30.135890 13650 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.138201 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:30.176193 11517 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:30.137630 11443 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:30.176622 11443 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:30.146912 11518 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:30.097060 11438 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
W20260709 13:15:30.087013 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.163791 12062 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:30.183943 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:30.184372 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.184580 11516 raft_consensus.cc:2376] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3. Re-sending same reply.
I20260709 13:15:30.150151 11442 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.156595 13444 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.149988 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:30.123306 13661 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.154735 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:30.081951 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:30.187845 13444 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.188076 13662 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.189208 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:30.189814 11513 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:30.155961 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 73053.5us
I20260709 13:15:30.150889 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:30.135504 13533 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:30.138041 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:30.163753 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:30.141003 11514 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:30.178323 12065 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:30.148936 13644 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:30.154242 13593 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.176406 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:30.155135 13297 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:30.164287 13421 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.159483 13348 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:30.156457 13647 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.136300 13640 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:30.191959 11409 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.199631 11443 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.127349 13594 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.219657 11364 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:30.043812 13646 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:30.200687 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46041.5us
I20260709 13:15:30.200460 11515 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:30.155679 13658 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:30.212548 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.185292 12062 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:15:30.212563 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.212826 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.213474 11335 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:30.254364 12065 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
W20260709 13:15:30.215348 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 65428.7us
I20260709 13:15:30.215965 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:30.217305 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.221963 13665 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.223629 13421 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:30.225205 13637 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.226809 11443 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.263150 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.232234 13644 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.216436 13297 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.212335 12061 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:30.199702 13533 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:30.265872 13544 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:30.276455 13544 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.276832 13544 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:30.261523 13665 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:30.277315 13665 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.234683 13661 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:30.242590 13647 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.243288 13664 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.212965 13229 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:30.248664 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.250542 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:30.256091 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.256574 12067 raft_consensus.cc:2410] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:30.207610 13353 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:30.138468 13654 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.276232 13533 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:30.280833 13649 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:15:30.278614 13594 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.246249 13639 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.271900 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:30.190706 13478 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.263053 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.263558 13426 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:30.266299 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:30.268602 11369 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:30.288964 11363 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:30.289698 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:30.270790 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54586.8us
I20260709 13:15:30.272308 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.241401 13607 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.282291 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:30.279676 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:30.260262 13626 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:30.283474 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.308400 11443 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:30.308346 11515 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:30.290887 12067 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:15:30.292927 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 91233.9us
I20260709 13:15:30.311513 11444 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.289464 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.304045 12069 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:30.278672 13297 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:30.352473 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:30.356141 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 76536.8us
I20260709 13:15:30.353029 13667 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:30.360322 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.361346 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.309943 11408 leader_election.cc:336] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.748s (SENT)
W20260709 13:15:30.362011 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 52099.1us
I20260709 13:15:30.293309 11510 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:30.363766 12033 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:15:30.304250 13669 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:15:30.366295 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:30.367755 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.292593 13533 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.291877 13654 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:30.290213 13353 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:30.300609 13639 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.288362 13664 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.374166 11369 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:30.373816 12067 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:30.284626 13681 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:30.289860 13498 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:30.376101 12033 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:30.417531 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:30.438570 12066 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.376776 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:30.377377 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.297405 13656 raft_consensus.cc:2764] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:15:30.311134 13607 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:30.448745 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 72724.5us
I20260709 13:15:30.381743 12069 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:30.383010 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.363129 13421 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.381093 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.387573 13584 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.387945 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.304333 13426 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:30.388693 13670 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:15:30.368258 13661 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:15:30.395473 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.396449 13675 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:15:30.404649 13647 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:30.363828 13680 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:30.406558 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.300669 13643 consensus_queue.cc:1175] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "01ae50c877d04ba687e4e06259ba5581" responder_term: 3 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:30.381539 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.410338 13544 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.410580 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:30.411105 13666 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:30.411801 13640 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:30.364073 13662 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:15:30.372838 11513 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
W20260709 13:15:30.443502 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 154095us
I20260709 13:15:30.544094 12066 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:30.433785 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:30.557363 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.557758 11516 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:30.434752 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:30.455328 12067 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:30.547477 11517 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:30.558641 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:30.558759 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.457561 11518 raft_consensus.cc:2368] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5.
I20260709 13:15:30.305394 13657 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:30.476810 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.561599 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:30.494128 11515 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:30.529181 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:30.437342 13681 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:30.436131 13664 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.457275 13607 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:30.534101 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 154528us
I20260709 13:15:30.419589 12063 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:30.415331 11365 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:30.458221 13654 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.460119 13584 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.458462 13678 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:30.459992 11361 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:30.491461 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 898) took 1108 ms. Trace:
I20260709 13:15:30.528626 13545 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:30.432711 13348 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:30.546954 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.552244 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 186270us
I20260709 13:15:30.426896 13665 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:30.590862 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.645371 12033 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:30.632043 11512 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:30.460247 13421 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:30.608100 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:30.578011 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.613057 11517 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:15:30.658259 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.545104 13661 consensus_queue.cc:1243] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:15:30.623929 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.498230 13650 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:30.605425 13658 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:30.665117 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.551615 13680 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:30.551930 13640 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.597894 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:30.627580 13584 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:30.658639 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.675782 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 80392.9us
I20260709 13:15:30.665844 12066 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:30.666998 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1746) took 1012 ms. Trace:
I20260709 13:15:30.625588 13581 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.674250 11517 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:30.562510 13626 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:30.663570 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 106253us
I20260709 13:15:30.647147 13545 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:30.711321 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:30.656529 13654 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:30.666327 13688 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.663723 13482 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:30.648991 13498 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:30.671532 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:30.674742 13640 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:30.601658 13353 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.646176 13593 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.675647 13658 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:30.714493 13648 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:30.630896 12064 rpcz_store.cc:276] 0709 13:15:29.352137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.353860 (+  1723us) service_pool.cc:224] Handling call
0709 13:15:30.437684 (+1083824us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54005d6954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:30.457149 (+ 19465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267548032}
I20260709 13:15:30.719671 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2146) took 1054 ms. Trace:
I20260709 13:15:30.722846 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:30.719116 11408 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.918s (SENT)
I20260709 13:15:30.728965 12069 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:30.869979 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:30.717767 13426 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:30.725841 11519 rpcz_store.cc:276] 0709 13:15:29.654767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.668116 (+ 13349us) service_pool.cc:224] Handling call
0709 13:15:29.668401 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:29.679656 (+ 11255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.679695 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:29.683177 (+  3482us) log.cc:844] Serialized 26 byte log entry
0709 13:15:30.098012 (+414835us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.098066 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.098132 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:30.666800 (+568668us) raft_consensus.cc:1685] finished
0709 13:15:30.666818 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.666942 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4865,"raft.queue_time_us":10997,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11062,"thread_start_us":153369,"threads_started":1,"wal-append.queue_time_us":495375}
I20260709 13:15:30.742641 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:30.751118 13682 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.346s	user 0.000s	sys 0.003s
I20260709 13:15:30.727990 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.728158 13646 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.724097 11514 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
W20260709 13:15:30.836608 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 175634us
I20260709 13:15:30.897957 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:30.751892 13297 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.726397 13679 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:15:30.856163 11437 rpcz_store.cc:276] 0709 13:15:29.665560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.673046 (+  7486us) service_pool.cc:224] Handling call
0709 13:15:29.682885 (+  9839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.696906 (+ 14021us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:30.112220 (+415314us) log.cc:844] Serialized 14 byte log entry
0709 13:15:30.719337 (+607117us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:30.719385 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:30.719443 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.719507 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.719619 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":268011,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":284366,"spinlock_wait_cycles":11279232,"thread_start_us":741747,"threads_started":3,"wal-append.queue_time_us":552127}
I20260709 13:15:30.871219 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.877913 13348 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:30.878965 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 134843us
I20260709 13:15:30.904731 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:30.829844 13581 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:30.829993 12063 raft_consensus.cc:2376] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3. Re-sending same reply.
W20260709 13:15:30.839672 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.747596 13697 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.135s
I20260709 13:15:30.901510 12066 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
W20260709 13:15:30.914875 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 195781us
W20260709 13:15:30.911384 11444 consensus_meta.cc:287] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.557s	user 0.005s	sys 0.001s
I20260709 13:15:30.915587 11444 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:30.901073 13482 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:30.900504 13594 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.605s	user 0.006s	sys 0.003s
I20260709 13:15:30.925526 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.924320 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.916422 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 381) took 1224 ms. Trace:
I20260709 13:15:30.926692 12069 raft_consensus.cc:2376] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5. Re-sending same reply.
I20260709 13:15:30.926813 11444 rpcz_store.cc:276] 0709 13:15:29.692226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.801448 (+109222us) service_pool.cc:224] Handling call
0709 13:15:30.282158 (+480710us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:30.916350 (+634192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15307008}
I20260709 13:15:30.930999 12066 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:30.933931 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:30.937187 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:30.912022 13544 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:30.914644 13646 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.946898 13678 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:30.916605 13657 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:30.948443 13594 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:30.918632 13426 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.935326 13348 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:30.906800 13545 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:30.955899 13698 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:15:30.966095 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.936303 11361 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:30.965482 13662 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:30.975574 13657 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:30.976959 13657 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:30.978693 13679 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:30.965041 13426 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.979444 13426 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.979817 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:30.980600 11365 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:30.982810 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:30.983430 11369 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.985083 11364 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:15:30.986258 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.964246 13581 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:30.987522 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:30.988202 11362 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:30.969484 13348 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:30.988934 13650 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:30.990190 13348 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:30.991012 13689 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:30.849532 13680 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.989943 13482 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.991766 13623 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.962905 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:30.994172 11363 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:30.829964 13688 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:30.995097 13693 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:15:30.996120 11362 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:15:30.998018 13686 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:15:30.998147 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:30.998339 13704 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:30.998782 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:30.999018 11262 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56588 (request call id 235) took 12723 ms (client timeout 15000 ms). Trace:
I20260709 13:15:30.971979 13658 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.999833 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:31.000526 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:31.001263 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:31.003683 13665 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:31.006899 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.036909 13648 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:31.007020 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:31.007571 13690 log.cc:927] Time spent T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:15:31.013146 11369 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:30.936183 13498 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.725440 13681 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:30.968720 13421 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.015204 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.016227 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.017563 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.014979 13657 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.019065 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:31.020821 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 116182us
W20260709 13:15:31.019868 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 297082us
I20260709 13:15:31.019629 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:31.025087 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.979027 13679 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:31.039460 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.040022 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:30.987562 13700 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.045143 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:31.051779 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.052088 11367 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:31.045944 13704 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:31.044718 13699 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 13:15:31.045701 12064 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:31.029870 13689 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:31.251284 11440 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:31.252676 11436 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:15:31.201828 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.232074 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 638) took 1545 ms. Trace:
I20260709 13:15:31.212668 11364 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:31.200412 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.250497 11437 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:31.252341 11442 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:31.255710 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.256098 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.257014 11443 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:31.258504 11483 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:31.259353 11363 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:31.225525 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.084450 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:31.260614 11370 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:31.178488 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.266614 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:31.267926 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:31.268537 11436 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:15:31.268903 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.160118 13705 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:15:31.269749 13721 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.079s	user 0.001s	sys 0.001s
I20260709 13:15:31.057070 13654 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:31.271584 13664 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.004s
I20260709 13:15:31.271332 13654 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.271970 13664 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.272586 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:31.158018 13691 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:15:31.273078 11441 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:31.273878 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:31.253804 12065 rpcz_store.cc:276] 0709 13:15:29.686283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.262049 (+575766us) service_pool.cc:224] Handling call
0709 13:15:30.279432 (+ 17383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:30.285548 (+  6116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.285587 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:30.285999 (+   412us) log.cc:844] Serialized 26 byte log entry
0709 13:15:30.601336 (+315337us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.601380 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.601442 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.159234 (+557792us) raft_consensus.cc:1685] finished
0709 13:15:31.159256 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.231999 (+ 72743us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2855,"raft.queue_time_us":27848,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":8099,"thread_start_us":235208,"threads_started":1,"wal-append.queue_time_us":392889}
W20260709 13:15:31.153823 13639 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.721s	user 0.006s	sys 0.003s
W20260709 13:15:31.277858 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 258300us
W20260709 13:15:31.175829 13607 consensus_meta.cc:287] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.548s	user 0.004s	sys 0.004s
I20260709 13:15:31.277928 12032 leader_election.cc:400] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5.
I20260709 13:15:31.222898 13707 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.280614 13636 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.280714 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:31.281519 13663 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.282012 13662 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:31.282248 13646 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.157668 13665 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.278908 13581 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.301951 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:31.184638 13688 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:31.199781 13444 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.283686 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:31.284307 12032 leader_election.cc:403] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:31.277328 13545 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.281458 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:31.164723 11365 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:31.066893 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.225126 13454 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:31.281260 13439 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:31.176316 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 475) took 1192 ms. Trace:
I20260709 13:15:31.263429 13617 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:31.263577 13704 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:31.302716 11516 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:31.195519 13593 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:31.046140 11262 rpcz_store.cc:269] 0709 13:15:18.286251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.286497 (+   246us) service_pool.cc:224] Handling call
0709 13:15:18.872590 (+586093us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b54001897d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc968090 00007f35bca06f3a 00007f35c27bd1cc 00007f35c28bb37e 00007f35c0d7c2a4 00007f35c0d7c232 00007f35c0d7c161 00007f35c0d7c0dc 00007f35c0d7b392 00007f35bad3948c 00007f35bad388c6
0709 13:15:28.948303 (+10075713us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b5400189628. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c0368161 00007f35c03f23af 00007f35c03f3b11 00007f35c28f42d1 00007f35c28f4b2f 00007f35c27b3d4b 00007f35c28bbc3e 00007f35c0d7c2a4 00007f35c0d7c232 00007f35c0d7c161 00007f35c0d7c0dc
0709 13:15:30.998949 (+2050646us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:29.033290 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:29.033995 (+   705us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:29.034050 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:15:29.034071 (+    21us) tablet.cc:662] Decoding operations
0709 13:15:29.034550 (+   479us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:29.034650 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:29.034679 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:15:29.035182 (+   503us) tablet.cc:701] Row locks acquired
0709 13:15:29.035232 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:15:29.035420 (+   188us) write_op.cc:270] Start()
0709 13:15:29.035583 (+   163us) write_op.cc:276] Timestamp: P: 1783602929035343 usec, L: 0
0709 13:15:29.035608 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:15:29.036410 (+   802us) log.cc:844] Serialized 907 byte log entry
0709 13:15:29.402214 (+365804us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35b98f40ac 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b
0709 13:15:30.631544 (+1229330us) op_driver.cc:464] REPLICATION: finished
0709 13:15:30.903685 (+272141us) write_op.cc:301] APPLY: starting
0709 13:15:30.903784 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:30.903923 (+   139us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:30.903959 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:30.950123 (+ 46164us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:30.992065 (+ 41942us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:15:30.992110 (+    45us) write_op.cc:312] APPLY: finished
0709 13:15:30.992382 (+   272us) log.cc:844] Serialized 29 byte log entry
0709 13:15:30.992558 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:30.992650 (+    92us) write_op.cc:454] Released schema lock
0709 13:15:30.992704 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:30.993083 (+   379us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:30.993111 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":143744768,"child_traces":[["op",{"apply.queue_time_us":238755,"apply.run_cpu_time_us":3156,"apply.run_wall_time_us":89401,"num_ops":2,"prepare.queue_time_us":45565,"prepare.run_cpu_time_us":40117,"prepare.run_wall_time_us":1049508,"raft.queue_time_us":569049,"raft.run_cpu_time_us":44505,"raft.run_wall_time_us":321508,"replication_time_us":1525231,"spinlock_wait_cycles":13115776,"thread_start_us":1333169,"threads_started":4,"wal-append.queue_time_us":893068}]]}
I20260709 13:15:31.300663 13662 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:31.064671 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:31.304495 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.163136 13426 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:31.311684 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 58219.8us
I20260709 13:15:31.348220 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:31.277940 13421 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:31.317468 13706 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:31.311188 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.318558 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1394) took 1084 ms. Trace:
I20260709 13:15:31.352506 11514 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:31.340461 13439 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:31.352452 11519 rpcz_store.cc:276] 0709 13:15:30.234498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.882407 (+647909us) service_pool.cc:224] Handling call
0709 13:15:30.896668 (+ 14261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:30.897216 (+   548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.897248 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:30.906760 (+  9512us) log.cc:844] Serialized 26 byte log entry
0709 13:15:31.064295 (+157535us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:31.064330 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.064395 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.318329 (+253934us) raft_consensus.cc:1685] finished
0709 13:15:31.318351 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.318498 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":146464,"threads_started":1,"wal-append.queue_time_us":363804}
I20260709 13:15:31.354205 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:31.354323 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.354828 11361 raft_consensus.cc:2376] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4. Re-sending same reply.
I20260709 13:15:31.355145 11519 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:31.356074 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.339965 13617 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.357586 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.358305 11519 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:31.358659 13617 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.293351 13607 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:31.320348 11511 raft_consensus.cc:2376] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5. Re-sending same reply.
I20260709 13:15:31.306885 13636 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.300004 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1392) took 1059 ms. Trace:
I20260709 13:15:31.311949 13581 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.316670 13639 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5.
I20260709 13:15:31.312222 13545 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.376219 13545 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.273768 13700 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.378652 13700 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.182914 13584 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.380959 13584 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:31.083817 13648 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.382918 12067 raft_consensus.cc:2368] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4.
I20260709 13:15:31.267315 13657 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:31.314584 11510 rpcz_store.cc:276] 0709 13:15:29.983832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.612049 (+628217us) service_pool.cc:224] Handling call
0709 13:15:30.670552 (+ 58503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:30.764076 (+ 93524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.825383 (+ 61307us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:30.903656 (+ 78273us) log.cc:844] Serialized 26 byte log entry
0709 13:15:30.972840 (+ 69184us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:30.972901 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.973960 (+  1059us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.165278 (+191318us) raft_consensus.cc:1685] finished
0709 13:15:31.175992 (+ 10714us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.176219 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18335,"raft.queue_time_us":181132,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":21007,"thread_start_us":62006,"threads_started":1,"wal-append.queue_time_us":133431}
I20260709 13:15:31.334800 11436 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:31.306046 13697 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:31.345311 11515 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:31.307641 13646 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.264591 13640 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:31.324795 13656 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:31.359570 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:31.319893 12061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 917) took 1787 ms (client timeout 2118 ms). Trace:
W20260709 13:15:31.401110 12061 rpcz_store.cc:269] 0709 13:15:29.532732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.680727 (+147995us) service_pool.cc:224] Handling call
0709 13:15:29.981589 (+300862us) spinlock_profiling.cc:243] Waited 905 us on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:31.319703 (+1338114us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x7b540046aad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:31.319830 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154689152}
I20260709 13:15:31.402489 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:31.315068 13707 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.403087 12061 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:31.403800 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.403087 13637 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.405174 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:31.352738 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.406069 11516 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:31.406833 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.360814 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1393) took 1126 ms. Trace:
I20260709 13:15:31.407717 11517 rpcz_store.cc:276] 0709 13:15:30.234271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.741968 (+507697us) service_pool.cc:224] Handling call
0709 13:15:30.747327 (+  5359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:31.032072 (+284745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:31.032103 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:31.039798 (+  7695us) log.cc:844] Serialized 26 byte log entry
0709 13:15:31.313315 (+273517us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:31.313355 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.313428 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.360631 (+ 47203us) raft_consensus.cc:1685] finished
0709 13:15:31.360649 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.360764 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4954,"raft.queue_time_us":165862,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":20914,"thread_start_us":426278,"threads_started":2,"wal-append.queue_time_us":300628}
I20260709 13:15:31.345866 12064 raft_consensus.cc:2410] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:31.409333 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:31.409965 12063 raft_consensus.cc:2376] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3. Re-sending same reply.
I20260709 13:15:31.410924 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:31.410990 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:31.411715 12064 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:31.412022 11438 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:31.412606 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:31.413391 12064 raft_consensus.cc:2410] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:31.413769 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:31.414222 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:31.365248 11483 leader_election.cc:336] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.671s (SENT)
I20260709 13:15:31.414824 12064 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:31.415081 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:31.415974 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50761.1us
I20260709 13:15:31.409348 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:31.417047 11517 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.417789 13640 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:31.418115 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:31.353284 13439 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.418403 13640 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:31.309329 13665 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:31.419531 13444 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:31.420105 13439 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.420295 13665 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.420047 13444 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.420763 13665 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.354717 13426 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.373203 13711 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.373672 13716 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.425022 13688 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:31.425269 13716 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.424618 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:31.373883 13581 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:31.385654 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:31.388067 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:31.389191 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:31.316797 13663 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.399120 11519 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:31.433151 11367 raft_consensus.cc:2393] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 2: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:31.407373 11516 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:31.346295 13662 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.422269 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.434190 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.434381 11515 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:31.387202 13697 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.368566 13706 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.366005 13623 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:31.425024 13711 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.317657 13710 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.078s
W20260709 13:15:31.435426 13687 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.550s	user 0.000s	sys 0.001s
I20260709 13:15:31.436991 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2152) took 1187 ms. Trace:
I20260709 13:15:31.438783 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.453150 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:31.449144 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:31.421190 13665 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:31.455277 11436 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:31.449355 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:31.453862 11445 rpcz_store.cc:276] 0709 13:15:30.249972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.250087 (+   115us) service_pool.cc:224] Handling call
0709 13:15:30.264476 (+ 14389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:30.264871 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.264898 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:30.265476 (+   578us) log.cc:844] Serialized 52 byte log entry
0709 13:15:30.550416 (+284940us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:30.603968 (+ 53552us) log.cc:844] Serialized 14 byte log entry
0709 13:15:30.722979 (+119011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.723043 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.436761 (+713718us) raft_consensus.cc:1685] finished
0709 13:15:31.436784 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.436931 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53913,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":4022,"raft.queue_time_us":265563,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":21324,"thread_start_us":403286,"threads_started":2,"wal-append.queue_time_us":390553}
I20260709 13:15:31.429072 12063 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:31.387693 13421 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:31.475536 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.478351 11445 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:31.478461 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:31.479204 11436 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:31.435485 13718 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.437920 13636 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.440269 13697 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:31.517999 11517 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:31.480105 13713 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:15:31.481321 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:31.399732 13646 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.437757 13656 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:31.484407 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:31.479262 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.523216 11445 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:31.523521 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:31.449478 13607 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.519043 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:31.524307 11519 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:31.493893 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:31.524652 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:31.524865 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:31.525643 11436 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:31.494107 13667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.131s	user 0.001s	sys 0.011s
I20260709 13:15:31.455704 13706 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:31.529094 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.535411 11445 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:31.536459 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:31.537470 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:31.529438 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:31.530917 11517 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:31.532079 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47734.9us
I20260709 13:15:31.483572 13439 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:31.531787 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.644292 11436 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
W20260709 13:15:31.532986 13667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.169s	user 0.001s	sys 0.012s
I20260709 13:15:31.532320 11365 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:31.526855 12064 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:31.530733 13636 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:31.367861 11512 rpcz_store.cc:276] 0709 13:15:30.233988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.716284 (+482296us) service_pool.cc:224] Handling call
0709 13:15:30.750642 (+ 34358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:30.750926 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.750955 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:30.751457 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:15:30.899351 (+147894us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.899414 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.900753 (+  1339us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.273681 (+372928us) raft_consensus.cc:1685] finished
0709 13:15:31.277108 (+  3427us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.292822 (+ 15714us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":129601,"threads_started":1,"wal-append.queue_time_us":271145}
I20260709 13:15:31.487555 11361 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:31.538295 13444 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.613030 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.344348 13679 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:31.533818 13722 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:15:31.649955 13667 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:31.651158 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 121758us
I20260709 13:15:31.651655 11519 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:15:31.535452 13454 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.580202 13689 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.657066 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:31.527083 13718 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.658120 11517 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:31.537093 13623 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
W20260709 13:15:31.665364 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.627594 13584 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:31.692672 13716 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.692665 13679 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:31.693590 11445 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:31.656901 13581 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:31.792339 11436 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:31.699267 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 169583us
I20260709 13:15:31.657449 13636 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:31.693496 13726 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:15:31.819075 13581 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:31.687018 13444 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:15:31.749074 13584 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.822156 13581 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:31.824220 13584 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:31.825207 12063 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:31.825934 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 499) took 1395 ms. Trace:
I20260709 13:15:31.778784 13711 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.827394 12061 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:31.827733 13706 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:31.828163 13663 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:31.776852 11438 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 2.
W20260709 13:15:31.938618 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 294635us
I20260709 13:15:32.027716 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:32.028271 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:32.029094 13637 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:32.029546 13678 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:32.030124 13678 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:32.030696 11519 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:31.884363 13665 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:31.809873 13454 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:31.971257 11510 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:15:32.030998 13678 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.031934 13646 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.032688 12061 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:32.034650 13706 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.034402 13667 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:31.948873 13656 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:32.035836 13657 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.036881 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:32.038012 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:32.001600 13348 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.023s	sys 0.025s
I20260709 13:15:32.042078 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:32.043725 11517 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:31.949616 13704 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.602s	user 0.006s	sys 0.003s
I20260709 13:15:31.907552 13636 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:31.857908 13654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.043s	sys 0.074s
I20260709 13:15:32.060384 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:32.043169 13662 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.008s
W20260709 13:15:32.059443 13348 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.024s	sys 0.026s
I20260709 13:15:32.055449 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 641) took 1697 ms. Trace:
I20260709 13:15:31.880371 12063 rpcz_store.cc:276] 0709 13:15:30.430394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.410651 (+980257us) service_pool.cc:224] Handling call
0709 13:15:31.825850 (+415199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:32.069818 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:32.063207 13584 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:32.064602 13138 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:32.072329 13138 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:32.062647 13654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.044s	sys 0.074s
I20260709 13:15:32.077144 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.077905 11517 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:32.078827 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.079524 11517 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:15:32.080390 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.080955 11517 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:32.084745 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.061348 11362 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:15:32.062162 13637 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:32.085500 11519 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:32.054217 11510 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:32.096401 11517 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:31.960860 13697 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.096508 11362 pending_rounds.cc:85] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 3
I20260709 13:15:32.112466 11362 pending_rounds.cc:107] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:15:32.112789 11362 raft_consensus.cc:2889] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:32.115015 11519 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:32.068966 13623 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:15:32.061973 13704 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:32.148166 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 102055us
I20260709 13:15:32.108757 13737 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.070346 13662 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.110939 13584 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.108458 13439 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:32.173636 11516 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:32.109102 13688 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:32.154031 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.149623 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:32.069880 12061 rpcz_store.cc:276] 0709 13:15:30.357739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.031905 (+1674166us) service_pool.cc:224] Handling call
0709 13:15:32.032286 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.053522 (+ 21236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.055351 (+  1829us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:32.073275 13138 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:32.174672 11336 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.038s	sys 0.058s
W20260709 13:15:32.182976 11336 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.039s	sys 0.058s
I20260709 13:15:32.147097 13684 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.183593 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:32.185046 13654 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:32.183951 12063 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:32.185519 13654 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:32.185846 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 502) took 1754 ms. Trace:
I20260709 13:15:32.185921 13654 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:32.186321 12063 rpcz_store.cc:276] 0709 13:15:30.431193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.098661 (+1667468us) service_pool.cc:224] Handling call
0709 13:15:32.146981 (+ 48320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.185581 (+ 38600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.185789 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:32.178817 13439 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.187619 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.188372 12063 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:32.179555 11512 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:15:32.193734 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44209.3us
I20260709 13:15:32.175827 13584 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.193616 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.197270 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.198559 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:32.199766 13731 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:15:32.198936 13662 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.201148 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1877) took 1157 ms. Trace:
I20260709 13:15:32.189339 13742 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:32.202526 13729 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:32.202989 13729 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.201557 11445 rpcz_store.cc:276] 0709 13:15:31.044199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.747437 (+703238us) service_pool.cc:224] Handling call
0709 13:15:31.747816 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.026170 (+278354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.026201 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.026704 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:15:32.096800 (+ 70096us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:32.166469 (+ 69669us) log.cc:844] Serialized 14 byte log entry
0709 13:15:32.167102 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.167199 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:32.200918 (+ 33719us) raft_consensus.cc:1685] finished
0709 13:15:32.200937 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.201091 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2690,"prepare.queue_time_us":91943,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":31,"raft.queue_time_us":281390,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":11925,"spinlock_wait_cycles":6831360,"thread_start_us":109338,"threads_started":2,"wal-append.queue_time_us":118959}
I20260709 13:15:32.189520 13637 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.170755 13737 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.170974 13656 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.199215 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 637) took 2513 ms. Trace:
I20260709 13:15:32.148180 13454 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.242875 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:32.202140 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 664717us
I20260709 13:15:32.251365 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:32.247771 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.252296 12069 raft_consensus.cc:2376] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3. Re-sending same reply.
I20260709 13:15:32.252581 12065 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 2.
I20260709 13:15:32.204524 11445 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:15:32.205256 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 402380us
W20260709 13:15:32.254328 12069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 508) took 1634 ms (client timeout 1967 ms). Trace:
W20260709 13:15:32.254737 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.254330 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 389) took 1200 ms. Trace:
I20260709 13:15:32.256731 11445 rpcz_store.cc:276] 0709 13:15:31.054244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.203871 (+1149627us) service_pool.cc:224] Handling call
0709 13:15:32.204234 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.254078 (+ 49844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.254278 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:32.255314 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:15:32.254722 12069 rpcz_store.cc:269] 0709 13:15:30.620464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.927781 (+307317us) service_pool.cc:224] Handling call
0709 13:15:32.242707 (+1314926us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:32.254262 (+ 11555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42559232}
I20260709 13:15:32.255138 12068 rpcz_store.cc:276] 0709 13:15:29.686070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.964239 (+278169us) service_pool.cc:224] Handling call
0709 13:15:30.079848 (+115609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:30.080274 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.080302 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:30.080735 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:15:30.122229 (+ 41494us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.122282 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.122364 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:30.636062 (+513698us) raft_consensus.cc:1685] finished
0709 13:15:30.636084 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.198935 (+1562851us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400522bd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:32.199140 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":923,"raft.queue_time_us":6027,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":8680,"spinlock_wait_cycles":265137920,"thread_start_us":32454,"threads_started":1,"wal-append.queue_time_us":138168,"wal-append.run_cpu_time_us":2961,"wal-append.run_wall_time_us":1446927}
I20260709 13:15:32.203370 13729 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:32.249358 13692 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.401s	user 0.005s	sys 0.021s
I20260709 13:15:32.266547 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.247771 13464 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:32.265195 13644 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:32.264977 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.267454 13739 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.267714 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:32.248039 13733 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:15:32.192656 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1879) took 1146 ms. Trace:
I20260709 13:15:32.248314 13677 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:15:32.267210 12069 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:32.264729 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:32.244681 13439 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.241961 13596 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.203776 13680 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:32.204700 13631 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:32.272200 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 391) took 1016 ms. Trace:
I20260709 13:15:32.278698 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.278918 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:32.290910 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.291046 11366 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:32.209365 13704 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:32.261605 13444 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:32.205024 13697 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.286547 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:32.288453 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:32.258720 13742 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:32.289500 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.314500 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:32.275772 13464 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:32.267653 13476 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.192328 13684 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.291688 11443 rpcz_store.cc:276] 0709 13:15:31.255514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.224187 (+968673us) service_pool.cc:224] Handling call
0709 13:15:32.238866 (+ 14679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.239106 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:32.239473 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:15:32.271859 (+ 32386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:32.271915 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:32.271961 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.272035 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.272149 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":194,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":93,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":9274,"thread_start_us":32031,"threads_started":1}
I20260709 13:15:32.265604 13637 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.294734 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.319667 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.293648 13444 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:32.320184 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 61051.2us
I20260709 13:15:32.315371 11367 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:32.324345 12069 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:32.315721 11445 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:15:32.331372 11408 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.691s (SENT)
I20260709 13:15:32.335340 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:32.344055 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 57595.4us
I20260709 13:15:32.317854 13704 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.317638 13229 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.352210 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:32.316458 13742 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.290319 13631 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.327809 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:32.356580 11367 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.344898 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:32.356978 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.357955 13758 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:15:32.363168 11367 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:32.326470 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 640) took 2394 ms. Trace:
I20260709 13:15:32.372982 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:32.364991 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:32.346165 13571 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:32.328609 13444 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:32.376004 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.376474 12033 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:32.355811 13739 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:32.345454 13584 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.374361 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.376842 11483 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:32.378477 11443 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:32.376993 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:32.386875 11366 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:32.387166 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:32.386875 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2154) took 1117 ms. Trace:
I20260709 13:15:32.385334 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1882) took 1034 ms. Trace:
I20260709 13:15:32.387826 11443 rpcz_store.cc:276] 0709 13:15:31.269955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.377866 (+1107911us) service_pool.cc:224] Handling call
0709 13:15:32.378157 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.386656 (+  8499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.386820 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:32.388123 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:32.388126 11445 rpcz_store.cc:276] 0709 13:15:31.351473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.384768 (+1033295us) service_pool.cc:224] Handling call
0709 13:15:32.385032 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.385147 (+   115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.385161 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:32.385187 (+    26us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:32.385204 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.385234 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.385296 (+    62us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:32.373034 13662 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.366042 13631 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:32.356150 12061 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:32.396790 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.392554 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:32.321856 13464 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.395958 13476 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:32.404871 13684 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:32.407388 13751 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.212s	user 0.002s	sys 0.000s
I20260709 13:15:32.408378 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1754) took 1115 ms. Trace:
I20260709 13:15:32.435144 13464 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.435456 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1878) took 1391 ms. Trace:
I20260709 13:15:32.356545 13229 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.425845 13476 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:32.428156 12068 raft_consensus.cc:2410] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:15:32.435982 12067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 501) took 2005 ms (client timeout 1704 ms). Trace:
I20260709 13:15:32.389925 13581 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:32.436866 13476 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.437279 13711 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:32.437696 13711 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.438035 13711 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:32.435885 11436 rpcz_store.cc:276] 0709 13:15:31.044459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.793217 (+748758us) service_pool.cc:224] Handling call
0709 13:15:31.815744 (+ 22527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.054210 (+238466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.054246 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.061846 (+  7600us) log.cc:844] Serialized 26 byte log entry
0709 13:15:32.149248 (+ 87402us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:32.149285 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.149349 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:32.406818 (+257469us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:32.435120 (+ 28302us) raft_consensus.cc:1685] finished
0709 13:15:32.435141 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.435337 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":233192,"raft.queue_time_us":241538,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":5068,"spinlock_wait_cycles":37556480,"thread_start_us":51585,"threads_started":1,"wal-append.queue_time_us":121008}
I20260709 13:15:32.441536 13707 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.442198 13762 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.434894 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:32.433516 13472 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:15:32.410635 13725 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:15:32.412882 13729 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.396291 13697 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:32.452385 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.452817 13718 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.432221 13596 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:32.452992 11515 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:32.275661 11438 rpcz_store.cc:276] 0709 13:15:31.044730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.038406 (+993676us) service_pool.cc:224] Handling call
0709 13:15:32.038790 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.039041 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:32.112341 (+ 73300us) log.cc:844] Serialized 14 byte log entry
0709 13:15:32.166452 (+ 54111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:32.173146 (+  6694us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:32.175662 (+  2516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.183270 (+  7608us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.191261 (+  7991us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":149531,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":117,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5666,"spinlock_wait_cycles":33534336,"thread_start_us":104403,"threads_started":2,"wal-append.queue_time_us":84540}
I20260709 13:15:32.453778 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.434998 11512 rpcz_store.cc:276] 0709 13:15:31.292581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.684884 (+392303us) service_pool.cc:224] Handling call
0709 13:15:31.768059 (+ 83175us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.195775 (+427716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.408085 (+212310us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54003d8954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:32.408310 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876027136}
I20260709 13:15:32.454279 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:32.454528 11519 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:32.455096 13644 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:32.455473 13644 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.455794 13644 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:32.452417 13697 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.456890 13697 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:32.436384 12067 rpcz_store.cc:269] 0709 13:15:30.430924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.558518 (+127594us) service_pool.cc:224] Handling call
0709 13:15:32.435789 (+1877271us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b5400741dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:32.435922 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893634816}
I20260709 13:15:32.457813 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:32.457983 11366 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:32.458359 12067 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:32.459851 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.448750 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.414235 13482 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:32.425959 13763 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.451282 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:32.452090 13757 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:15:32.442719 13762 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.462428 12069 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:32.422282 13662 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.478678 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.463702 13482 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:32.480607 12066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 922) took 1820 ms (client timeout 2231 ms). Trace:
I20260709 13:15:32.478777 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:32.482208 12066 rpcz_store.cc:269] 0709 13:15:30.660467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.478403 (+1817936us) service_pool.cc:224] Handling call
0709 13:15:32.480546 (+  2143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:32.463289 13581 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:32.483191 13482 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.483075 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.482905 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:32.481525 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:32.488335 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 118595us
I20260709 13:15:32.457839 13636 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.478473 13680 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:32.492790 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.469050 13584 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:15:32.375306 12062 rpcz_store.cc:276] 0709 13:15:29.928447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.276555 (+348108us) service_pool.cc:224] Handling call
0709 13:15:30.276849 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.304443 (+ 27594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.304477 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:30.304523 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:30.304554 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.293758 (+1989204us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b5400751054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:32.313540 (+ 19782us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.322541 (+  9001us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63011712}
I20260709 13:15:32.493637 12067 raft_consensus.cc:2410] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:32.495996 11366 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:32.495327 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.494132 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.499760 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.497216 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.502924 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.503530 12067 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:32.505033 11367 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:32.500769 13138 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:32.492736 13737 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:32.481645 13763 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.494832 13728 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:15:32.485673 13739 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.673305 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.510375 11367 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
W20260709 13:15:32.639460 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.498167 13740 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.678428 12069 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:32.674532 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:32.612450 11483 leader_election.cc:336] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.231s (SENT)
I20260709 13:15:32.687541 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.691347 12067 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:15:32.678323 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:32.691169 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:32.631354 11408 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.989s (SENT)
W20260709 13:15:32.683287 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:32.692886 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 61588us
W20260709 13:15:32.695091 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.704388 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:32.670881 13768 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:32.585906 13596 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.522634 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.509786 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:32.505777 13454 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:32.558465 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:32.526757 13229 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:32.618062 13762 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.559887 13662 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.529502 13636 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:32.603928 13571 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:32.725580 13774 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:32.671960 13476 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.625455 13631 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.598409 13444 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.630962 13697 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.723376 11366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:32.703594 13740 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.713534 13768 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.511822 13581 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:32.778862 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 166457us
I20260709 13:15:32.779863 11366 raft_consensus.cc:606] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:15:32.780308 11366 raft_consensus.cc:614] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:32.723362 13728 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:15:32.535771 13680 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.685678 13138 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.714542 13769 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:32.711958 13596 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:32.778015 13654 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:15:32.794283 11366 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:32.795028 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:32.795885 11514 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:32.797052 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:32.797331 11333 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:32.799459 13596 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.793062 13704 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:32.793233 13742 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.802867 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:32.803581 12061 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.804586 13476 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.805743 12062 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:32.806391 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 647) took 1459 ms. Trace:
I20260709 13:15:32.806741 12062 rpcz_store.cc:276] 0709 13:15:31.347307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.782087 (+1434780us) service_pool.cc:224] Handling call
0709 13:15:32.805370 (+ 23283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.806122 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.806332 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:32.807885 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1395) took 1527 ms (client timeout 1691 ms). Trace:
W20260709 13:15:32.808292 11511 rpcz_store.cc:269] 0709 13:15:31.280966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.281192 (+   226us) service_pool.cc:224] Handling call
0709 13:15:32.807734 (+1526542us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b54000672d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:32.807829 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31219968}
I20260709 13:15:32.809029 13768 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.809219 13662 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:32.809443 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:32.812281 11408 leader_election.cc:336] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.884s (SENT)
I20260709 13:15:32.812650 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:32.782708 13636 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.787777 13769 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.721688 13667 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:15:32.810072 11511 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:32.809767 13662 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.734228 13404 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 13:15:32.824141 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 101421us
I20260709 13:15:32.780401 13697 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.797727 13763 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.827709 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:32.797417 13637 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.809617 13654 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.764871 13571 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.815793 13720 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:32.822338 13742 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.831751 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:32.833349 12069 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:32.736019 13644 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:32.822540 13769 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.824728 13636 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.849025 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:33.033339 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:33.034510 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:33.035275 11511 raft_consensus.cc:2410] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:15:33.067643 11408 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.063s (SENT) [suppressed 33 similar messages]
I20260709 13:15:33.108884 13654 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:33.112682 13482 consensus_meta.cc:287] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.629s	user 0.004s	sys 0.004s
I20260709 13:15:33.113149 13482 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.115257 11513 raft_consensus.cc:2393] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:33.116518 13662 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:32.836439 11512 raft_consensus.cc:2410] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:33.154722 13571 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.155784 13404 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.156347 13404 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:33.158660 13404 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.021667 12061 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:33.399184 11483 leader_election.cc:336] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.577s (SENT)
W20260709 13:15:33.397614 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 569841us
I20260709 13:15:33.275835 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:33.472553 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:33.473896 12068 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:33.180303 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.273948 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 206321us
I20260709 13:15:32.855603 13763 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:15:33.474511 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 75371.5us
I20260709 13:15:33.106477 13444 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:32.902534 12063 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:33.478031 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:32.834959 13720 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.838965 13740 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:32.879559 13711 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:32.873528 13667 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:33.475312 13631 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.175429 13776 raft_consensus.cc:2764] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:15:33.479054 13763 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.478639 13773 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:32.902887 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:33.479800 13404 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.492512 13644 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:33.485060 13720 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.487630 13444 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.500461 12068 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:33.501612 13404 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.496541 13138 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.498145 13763 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:33.512566 11514 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:33.515913 12068 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:33.516796 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:33.521951 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 650) took 1802 ms. Trace:
I20260709 13:15:33.522526 12068 rpcz_store.cc:276] 0709 13:15:31.715985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.514116 (+1798131us) service_pool.cc:224] Handling call
0709 13:15:33.514969 (+   853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.518210 (+  3241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.518426 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:33.502949 12062 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:33.504187 13776 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.512801 13711 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:33.515143 13444 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:33.523727 12061 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:33.518273 13773 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:33.528430 13769 consensus_meta.cc:287] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.653s	user 0.009s	sys 0.000s
I20260709 13:15:33.537490 12068 pending_rounds.cc:85] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 5
I20260709 13:15:33.540033 12068 pending_rounds.cc:107] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:15:33.540431 12068 raft_consensus.cc:2889] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:33.547255 13776 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.551765 13404 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.554466 13571 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.556232 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 933) took 1374 ms. Trace:
I20260709 13:15:33.556679 12063 rpcz_store.cc:276] 0709 13:15:32.181534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.197935 (+ 16401us) service_pool.cc:224] Handling call
0709 13:15:33.555989 (+1358054us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54000587d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:33.556162 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553762560}
I20260709 13:15:33.559607 13776 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.561738 12067 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:33.523458 13667 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:15:33.561424 13138 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.575198 13138 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.577330 13138 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.577562 13634 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.555101 13571 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.579685 13571 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:33.582283 13421 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.584483 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:33.585153 11439 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.550945 13636 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:33.538275 13654 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.588508 13654 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.590739 13654 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.592412 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 509) took 2949 ms. Trace:
I20260709 13:15:33.592787 12069 rpcz_store.cc:276] 0709 13:15:30.642824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.703347 (+2060523us) service_pool.cc:224] Handling call
0709 13:15:32.712777 (+  9430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.838672 (+125895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.592184 (+753512us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400741dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:33.592349 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276816256}
I20260709 13:15:33.570794 13781 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:15:33.596374 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.597765 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.597911 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.598431 11445 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:33.596382 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.599237 13644 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.562331 13769 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.599727 13644 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.601486 11436 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:33.601840 11436 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:33.547748 13773 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.574450 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 649) took 1904 ms. Trace:
I20260709 13:15:33.603487 11436 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.604362 12061 rpcz_store.cc:276] 0709 13:15:31.670084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.486798 (+1816714us) service_pool.cc:224] Handling call
0709 13:15:33.500921 (+ 14123us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.535453 (+ 34532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.573453 (+ 38000us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:33.605001 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 491) took 3579 ms. Trace:
I20260709 13:15:33.605607 13773 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.606516 13297 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.601526 13769 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.607673 11440 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:33.608173 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.608740 11445 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:33.607777 13444 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:33.605424 12064 rpcz_store.cc:276] 0709 13:15:30.026017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.423908 (+1397891us) service_pool.cc:224] Handling call
0709 13:15:31.465023 (+ 41115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:31.657567 (+192544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.604735 (+1947168us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b5400693820. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc975dc0 00007f35bca01a02 00007f35bc9fddd6 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:33.604938 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169361280}
I20260709 13:15:33.611091 12064 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:33.609293 13688 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.611827 13688 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.612080 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 937) took 1429 ms. Trace:
I20260709 13:15:33.612478 12064 rpcz_store.cc:276] 0709 13:15:32.182428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.610426 (+1427998us) service_pool.cc:224] Handling call
0709 13:15:33.610734 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.611490 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.612008 (+   518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:33.613783 12069 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:33.614347 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 934) took 1432 ms. Trace:
I20260709 13:15:33.614953 13769 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.615109 13297 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.615525 13769 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.615625 13297 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.601802 13644 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.617592 13769 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:33.618706 11441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1862) took 2612 ms (client timeout 1926 ms). Trace:
W20260709 13:15:33.624054 11441 rpcz_store.cc:269] 0709 13:15:31.006649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.006787 (+   138us) service_pool.cc:224] Handling call
0709 13:15:33.618511 (+2611724us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540028a5d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:33.618644 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502090496}
I20260709 13:15:33.604847 11436 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.604357 13662 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.621908 13763 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:33.615939 13787 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:15:33.614648 12069 rpcz_store.cc:276] 0709 13:15:32.181750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.593800 (+1412050us) service_pool.cc:224] Handling call
0709 13:15:33.613459 (+ 19659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.614139 (+   680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.614301 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:33.629990 13649 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.617597 13297 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.630038 13662 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.631012 13644 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.632912 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:33.633337 12061 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:33.634173 13720 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.638779 11436 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:33.644095 12069 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:33.644776 13439 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:33.646898 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 130156us
I20260709 13:15:33.647053 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:33.647706 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:33.654613 12035 connection.cc:256] RPC call timeout handler was delayed by -1.791148303s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:15:33.643567 11439 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:33.658334 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:33.659185 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:33.659812 12069 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:33.645303 13644 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.660584 13695 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:33.661386 13752 log.cc:927] Time spent T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.464s	user 0.008s	sys 0.006s
I20260709 13:15:33.663970 12062 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
W20260709 13:15:33.664167 13780 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:15:33.665934 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1769) took 1161 ms. Trace:
W20260709 13:15:33.655577 13678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.452s	user 0.013s	sys 0.027s
I20260709 13:15:33.649534 13421 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:15:33.666253 11511 rpcz_store.cc:276] 0709 13:15:32.504179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.223582 (+719403us) service_pool.cc:224] Handling call
0709 13:15:33.267191 (+ 43609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.302928 (+ 35737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.302971 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.429274 (+126303us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.549448 (+120174us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:33.549500 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.549589 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.665728 (+116139us) raft_consensus.cc:1685] finished
0709 13:15:33.665748 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.665880 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":142,"thread_start_us":61103,"threads_started":1,"wal-append.queue_time_us":68874}
I20260709 13:15:33.669497 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:33.643147 13649 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.685328 13644 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:33.675798 13661 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.686434 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2061) took 1261 ms. Trace:
W20260709 13:15:33.667773 11439 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1870) took 2640 ms (client timeout 1925 ms). Trace:
I20260709 13:15:33.686745 12065 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:33.672785 13662 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.673449 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:33.672837 13720 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.659458 13654 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:15:33.675028 13439 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.668169 12034 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:33.675442 13482 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.681360 13711 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:33.682004 13685 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 2.940s	user 0.006s	sys 0.028s
W20260709 13:15:33.683151 11516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1402) took 2338 ms (client timeout 2064 ms). Trace:
I20260709 13:15:33.683260 11515 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:33.650694 13626 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:15:33.684001 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1771) took 1010 ms. Trace:
I20260709 13:15:33.683178 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:33.681942 11408 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.064s (SENT)
I20260709 13:15:33.687896 12061 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:33.688727 13678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.485s	user 0.014s	sys 0.028s
I20260709 13:15:33.690689 13711 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:33.691973 11516 rpcz_store.cc:269] 0709 13:15:31.344370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.433949 (+ 89579us) service_pool.cc:224] Handling call
0709 13:15:33.682944 (+2248995us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b540031ef54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:33.683091 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1693447424}
I20260709 13:15:33.698730 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:33.699138 13571 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.699445 11516 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 3: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:33.699734 13571 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.700899 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:33.701460 11516 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.702042 13571 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:33.686470 13649 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:33.689445 11439 rpcz_store.cc:269] 0709 13:15:31.026956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.028911 (+  1955us) service_pool.cc:224] Handling call
0709 13:15:33.584362 (+2555451us) spinlock_profiling.cc:243] Waited 7.48 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9eee 0000000000375939 00007f35c194d1e0 00007f35c1a160c4 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:33.666438 (+ 82076us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17201536}
I20260709 13:15:33.704762 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.705540 11363 raft_consensus.cc:2368] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5.
W20260709 13:15:33.705791 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.710867 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 645) took 2434 ms. Trace:
I20260709 13:15:33.713660 13720 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:33.714035 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.714196 13720 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.715072 13763 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.715746 13764 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:33.699776 13626 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:33.716267 13720 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.716043 13688 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:33.695359 11514 rpcz_store.cc:276] 0709 13:15:32.673742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.796780 (+123038us) service_pool.cc:224] Handling call
0709 13:15:33.683818 (+887038us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54000668d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:33.683947 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574305152}
I20260709 13:15:33.711278 12067 rpcz_store.cc:276] 0709 13:15:31.275972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.701185 (+1425213us) service_pool.cc:224] Handling call
0709 13:15:33.561550 (+860365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.562160 (+   610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.710621 (+148461us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54000587d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:33.710807 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1977936256}
I20260709 13:15:33.705933 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2057) took 1315 ms. Trace:
I20260709 13:15:33.719254 11363 rpcz_store.cc:276] 0709 13:15:32.390799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.704476 (+1313677us) service_pool.cc:224] Handling call
0709 13:15:33.705879 (+  1403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:33.686863 11366 rpcz_store.cc:276] 0709 13:15:32.425226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.784655 (+359429us) service_pool.cc:224] Handling call
0709 13:15:32.784982 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.794658 (+  9676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.686164 (+891506us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b5400422f54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:33.686372 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20887424}
I20260709 13:15:33.719811 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.716686 13626 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:33.716120 13764 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:33.722364 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:33.692950 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1391) took 3647 ms. Trace:
I20260709 13:15:33.693897 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:33.729094 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47167.5us
I20260709 13:15:33.721422 11516 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:33.688572 12069 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:33.665122 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:33.730068 11408 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.102s (SENT)
I20260709 13:15:33.732542 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.732736 11408 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:33.727039 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 58955.4us
I20260709 13:15:33.731724 13790 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:33.726209 13764 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.731956 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:33.742525 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:33.743285 11518 rpcz_store.cc:276] 0709 13:15:30.045932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.610468 (+564536us) service_pool.cc:224] Handling call
0709 13:15:30.626501 (+ 16033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:30.722737 (+ 96236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.722765 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.560019 (+2837254us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b2c000fe208. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:33.560407 (+   388us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.643640 (+ 83233us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:33.643672 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.644291 (+   619us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.676686 (+ 32395us) raft_consensus.cc:1685] finished
0709 13:15:33.676708 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.692894 (+ 16186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41370,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":38230,"spinlock_wait_cycles":13453184,"thread_start_us":169265,"threads_started":2,"wal-append.queue_time_us":109008}
W20260709 13:15:33.694944 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 47314.8us
I20260709 13:15:33.734926 12067 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:33.735486 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:33.763643 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:33.763718 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:33.721526 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.736330 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:33.738870 11363 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:15:33.723663 13694 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 2.895s	user 0.006s	sys 0.015s
I20260709 13:15:33.741169 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:33.765797 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:33.739048 13679 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.767709 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.745182 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 928) took 1571 ms. Trace:
I20260709 13:15:33.739590 13790 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.734181 13348 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:33.773384 13643 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:33.725063 13626 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.740850 13788 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.725112 13664 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.774121 13679 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:33.732545 12062 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:33.775055 13790 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.748330 13764 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.774593 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:33.776087 13661 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:33.776475 13661 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.776973 13661 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.768852 13426 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.778939 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:33.780176 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:33.780320 11513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 500) took 1327 ms (client timeout 1633 ms). Trace:
W20260709 13:15:33.780712 11513 rpcz_store.cc:269] 0709 13:15:32.453373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.453527 (+   154us) service_pool.cc:224] Handling call
0709 13:15:33.780122 (+1326595us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b54000ea954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:33.780265 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519483776}
I20260709 13:15:33.747442 13716 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:33.750563 13711 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:33.752532 13688 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:33.753326 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.798581 11364 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:33.799016 11442 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:33.776183 13679 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:33.754698 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.799376 13761 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:33.800482 13348 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:33.801003 13348 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:33.731179 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:33.802206 11516 raft_consensus.cc:2368] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:33.801932 13348 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.803056 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:33.767472 13662 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:33.767706 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:33.768049 13741 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.769348 11369 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:33.771303 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:33.814584 11444 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:15:33.815240 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44037.4us
I20260709 13:15:33.774863 13643 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:33.781749 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:33.780718 11511 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:33.782231 11519 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:15:33.750020 12060 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:33.785632 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.785071 12061 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:33.770694 13678 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.761497 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:33.809839 11516 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:33.765995 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 927) took 1592 ms. Trace:
I20260709 13:15:33.782590 13716 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:33.821642 12068 rpcz_store.cc:276] 0709 13:15:32.173907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.524032 (+1350125us) service_pool.cc:224] Handling call
0709 13:15:33.537254 (+ 13222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.541454 (+  4200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.541489 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.541970 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.646067 (+104097us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:33.743912 (+ 97845us) log.cc:844] Serialized 14 byte log entry
0709 13:15:33.765701 (+ 21789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.765774 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.765815 (+    41us) raft_consensus.cc:1685] finished
0709 13:15:33.765827 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.765938 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":96577,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":650,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":191775,"threads_started":2,"wal-append.queue_time_us":131439}
W20260709 13:15:33.816780 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 53130.3us
I20260709 13:15:33.816937 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:33.818954 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:33.821300 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 59853.4us
I20260709 13:15:33.825054 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:33.809615 13426 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:15:33.826395 13426 consensus_queue.cc:1243] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:33.826767 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.826997 13688 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:33.827673 11437 raft_consensus.cc:2410] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:33.820585 13634 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.772420 12063 rpcz_store.cc:276] 0709 13:15:32.174163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.557658 (+1383495us) service_pool.cc:224] Handling call
0709 13:15:33.558062 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:33.558530 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.558573 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:33.559208 (+   635us) log.cc:844] Serialized 52 byte log entry
0709 13:15:33.646548 (+ 87340us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:33.735823 (+ 89275us) log.cc:844] Serialized 14 byte log entry
0709 13:15:33.738733 (+  2910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.738802 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.738845 (+    43us) raft_consensus.cc:1685] finished
0709 13:15:33.738858 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.745120 (+  6262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2499,"prepare.queue_time_us":91073,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":19765,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":28694,"spinlock_wait_cycles":103374208,"thread_start_us":109132,"threads_started":2,"wal-append.queue_time_us":90695}
I20260709 13:15:33.827167 11514 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:33.830075 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1770) took 1228 ms. Trace:
I20260709 13:15:33.830410 11514 rpcz_store.cc:276] 0709 13:15:32.601586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.805471 (+1203885us) service_pool.cc:224] Handling call
0709 13:15:33.825644 (+ 20173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.829829 (+  4185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.830020 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:33.830896 13716 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:33.831534 13679 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:33.828111 13709 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:15:33.832619 13709 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:33.832949 13709 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:33.833431 13662 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:33.833662 13709 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:33.834586 13661 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.827696 13688 consensus_queue.cc:1243] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:33.820097 13641 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.826829 13643 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:33.836408 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:33.836549 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:33.837699 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:33.823860 13711 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:33.840507 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.864820 11369 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:33.865805 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.820050 13741 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:33.842165 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:33.846786 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:33.834807 13709 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.858899 11334 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:33.842279 13679 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:33.841449 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:33.869493 12067 raft_consensus.cc:2410] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:33.839984 11513 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:33.867604 11444 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:33.870774 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1776) took 1053 ms. Trace:
I20260709 13:15:33.871084 11513 rpcz_store.cc:276] 0709 13:15:32.817874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.837078 (+1019204us) service_pool.cc:224] Handling call
0709 13:15:33.839779 (+  2701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.870550 (+ 30771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.870728 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:33.872453 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.873183 11363 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:33.868316 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:33.876397 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:33.879786 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2056) took 1489 ms. Trace:
I20260709 13:15:33.880147 11367 rpcz_store.cc:276] 0709 13:15:32.390218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.717593 (+327375us) service_pool.cc:224] Handling call
0709 13:15:32.717932 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.811726 (+ 93794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.811755 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:32.811792 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:32.811821 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.879587 (+1067766us) spinlock_profiling.cc:243] Waited 93.6 ms on lock 0x7b5400423954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:33.879630 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.879736 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215292800}
I20260709 13:15:33.882786 11483 leader_election.cc:400] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5.
I20260709 13:15:33.883140 11483 leader_election.cc:403] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:33.862121 13643 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.864647 13711 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.908154 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:33.868566 13679 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.864523 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:33.899698 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:33.933598 11367 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.890744 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.847678 13657 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:15:33.909729 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 73246.9us
I20260709 13:15:33.937515 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:33.892570 11408 leader_election.cc:400] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:33.941460 11408 leader_election.cc:403] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:15:33.872257 13776 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:33.859838 13712 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:15:33.857226 13426 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:15:33.872025 13761 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:33.887895 13619 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:33.877394 13644 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.874531 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2043) took 2862 ms. Trace:
I20260709 13:15:33.898516 13649 raft_consensus.cc:2764] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:15:33.898957 13678 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5.
I20260709 13:15:33.864382 13641 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:33.910382 13711 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:33.907500 13643 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:33.948959 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 74725.4us
I20260709 13:15:33.877056 13656 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:15:33.887748 13634 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:15:33.932075 13679 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:33.952188 11439 pending_rounds.cc:85] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 4
I20260709 13:15:33.966888 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:33.945312 13776 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.957234 13761 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:33.968690 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 87587.9us
I20260709 13:15:33.969866 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:33.962491 13781 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:33.963073 13348 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:33.960417 11362 rpcz_store.cc:276] 0709 13:15:31.005797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.155662 (+149865us) service_pool.cc:224] Handling call
0709 13:15:31.177997 (+ 22335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.113371 (+935374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.855092 (+1741721us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54000c6f54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:33.865684 (+ 10592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950970112}
I20260709 13:15:33.966156 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2045) took 2622 ms. Trace:
I20260709 13:15:33.966663 13619 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:33.989969 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.976590 11439 pending_rounds.cc:107] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:15:33.973912 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:33.982954 13776 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:33.983126 13761 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:34.019933 11367 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:33.984930 13348 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.995671 11439 raft_consensus.cc:2889] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:34.027541 13348 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:34.070997 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.071733 11436 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.030408 13664 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:34.076444 13802 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:15:34.077044 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1768) took 1577 ms. Trace:
I20260709 13:15:34.077473 11519 rpcz_store.cc:276] 0709 13:15:32.499826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.499966 (+   140us) service_pool.cc:224] Handling call
0709 13:15:33.781988 (+1282022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.819490 (+ 37502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.076829 (+257339us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54000ea950. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:34.076986 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2924183936}
W20260709 13:15:34.078469 13800 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:15:34.078840 11519 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:34.060699 13661 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:34.075162 13803 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:15:34.077934 13799 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:15:33.985610 11365 rpcz_store.cc:276] 0709 13:15:31.343355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.345811 (+  2456us) service_pool.cc:224] Handling call
0709 13:15:31.353349 (+  7538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:31.651119 (+297770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.958242 (+2307123us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000a73d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:33.963477 (+  5235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283474304}
I20260709 13:15:34.066419 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:34.143633 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 176784us
I20260709 13:15:34.144790 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:34.073933 13664 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:34.151517 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.160012 13809 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:15:34.168201 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.150556 13664 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.202188 11511 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:34.162689 13454 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.175853 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:34.150525 11362 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
W20260709 13:15:34.206967 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 187718us
W20260709 13:15:34.237203 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.122789 13619 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:34.268673 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.276836 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.234560 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:34.234993 13426 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.290601 12062 consensus_meta.cc:287] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.007s
I20260709 13:15:34.237461 13818 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:34.298451 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 63484.7us
I20260709 13:15:34.286806 13679 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.296617 11436 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:34.323167 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:34.418154 13661 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.418700 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:34.418962 13696 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.419462 13696 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.420073 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:34.422148 13696 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:34.334167 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:34.417672 12032 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:34.336944 13643 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.309429 13716 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:15:34.236811 13454 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:34.291169 13807 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:15:34.307363 11365 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.339757 11361 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:34.307647 13426 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:34.376549 13764 consensus_meta.cc:287] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.601s	user 0.003s	sys 0.014s
I20260709 13:15:34.378144 13818 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:34.379871 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1772) took 1694 ms. Trace:
I20260709 13:15:34.295437 12062 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:34.392527 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:34.398413 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 253663us
I20260709 13:15:34.416816 13805 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.424552 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.496888 11445 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:34.397261 13826 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:34.455789 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:34.464413 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.466151 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.499223 11369 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
W20260709 13:15:34.463544 13790 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.017s
I20260709 13:15:34.489095 11518 rpcz_store.cc:276] 0709 13:15:32.685033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.823913 (+1138880us) service_pool.cc:224] Handling call
0709 13:15:33.841474 (+ 17561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.841673 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.841697 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.858116 (+ 16419us) log.cc:844] Serialized 26 byte log entry
0709 13:15:34.028354 (+170238us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:34.028386 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.072514 (+ 44128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.379646 (+307132us) raft_consensus.cc:1685] finished
0709 13:15:34.379666 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.379813 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":129738,"threads_started":1,"wal-append.queue_time_us":369566}
I20260709 13:15:34.466935 13664 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:34.502147 11436 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:15:34.548957 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.549533 11436 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:34.552273 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48853.9us
I20260709 13:15:34.499703 11443 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:34.484586 13715 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.418679 13661 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.501034 13667 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.584062 11364 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:15:34.580341 11409 leader_election.cc:400] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4.
I20260709 13:15:34.592731 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.597399 11443 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:34.597555 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.592317 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 136609us
I20260709 13:15:34.593582 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.503135 13826 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.594228 11445 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:34.581187 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.599056 11437 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:34.599664 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.599972 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.600472 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.600584 11437 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:34.600960 11445 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:34.580629 11436 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:34.601677 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.602092 11436 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:34.602411 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:34.609498 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 191918us
I20260709 13:15:34.611876 11436 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:15:34.615547 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.628787 11442 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:34.545957 13805 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.643080 11437 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
W20260709 13:15:34.623121 13711 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.652s	user 0.004s	sys 0.008s
I20260709 13:15:34.603348 13826 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.656875 11443 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:34.552742 13790 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.630308 13679 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:34.770323 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.660496 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.752063 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.592861 13833 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:15:34.736593 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.780369 11442 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
W20260709 13:15:34.780296 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 201660us
W20260709 13:15:34.644774 13781 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.001s
I20260709 13:15:34.781044 11436 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:34.783073 13664 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:34.794553 13781 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:34.798779 13454 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:34.800155 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:34.804479 13426 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:34.805696 13821 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:34.806180 13641 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:34.807312 13478 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.805675 13454 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:34.808633 13834 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 13:15:34.672652 13818 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.808081 13454 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.788822 13654 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.811848 13654 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.813480 12068 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:34.814785 13654 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:34.814791 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:34.816180 12068 raft_consensus.cc:2376] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3. Re-sending same reply.
I20260709 13:15:34.816485 13805 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:34.817101 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:34.819876 11445 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 4 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:15:34.821313 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 221661us
W20260709 13:15:34.746881 13827 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 13:15:34.771840 13828 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:15:34.712082 13761 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.655s	user 0.004s	sys 0.009s
I20260709 13:15:34.760735 13644 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:34.735772 13716 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.819826 11437 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:34.846527 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:34.771171 13643 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:34.826553 11409 connection.cc:256] RPC call timeout handler was delayed by -2.8916964110001118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 13:15:34.753407 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:34.854009 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.854691 11437 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:34.851344 11409 connection.cc:256] RPC call timeout handler was delayed by -2.6981964110000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
I20260709 13:15:34.861061 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.862407 11437 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:34.864667 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.866838 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.868813 11437 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:34.875527 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.876730 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.877372 11437 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:34.660553 13711 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:34.885927 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:34.889194 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:34.887343 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:34.890805 12068 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:34.864390 13817 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
I20260709 13:15:34.896395 13661 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:34.902212 11512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1763) took 2411 ms (client timeout 2667 ms). Trace:
W20260709 13:15:34.902765 11512 rpcz_store.cc:269] 0709 13:15:32.490570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.492376 (+  1806us) service_pool.cc:224] Handling call
0709 13:15:34.896409 (+2404033us) spinlock_profiling.cc:243] Waited 96.1 ms on lock 0x7b540031f454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:34.902140 (+  5731us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220944768}
I20260709 13:15:34.904532 11365 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:34.906386 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 658) took 1266 ms. Trace:
I20260709 13:15:34.906746 12065 rpcz_store.cc:276] 0709 13:15:33.639559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.639665 (+   106us) service_pool.cc:224] Handling call
0709 13:15:34.906224 (+1266559us) spinlock_profiling.cc:243] Waited 39 ms on lock 0x7b5400509954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:34.906330 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89635712}
I20260709 13:15:34.907471 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.908895 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.909509 11438 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:34.910439 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:34.910926 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.913833 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.914479 11437 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:34.911224 11438 raft_consensus.cc:2410] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:15:34.897382 13644 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:34.916134 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.916405 13644 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:34.917423 12060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 914) took 5385 ms (client timeout 1671 ms). Trace:
W20260709 13:15:34.917903 12060 rpcz_store.cc:269] 0709 13:15:29.531802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.565333 (+ 33531us) service_pool.cc:224] Handling call
0709 13:15:34.917205 (+5351872us) spinlock_profiling.cc:243] Waited 4.16 s on lock 0x7b5400595bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:34.917323 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9576290560}
I20260709 13:15:34.919457 13756 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:34.919857 13756 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:34.920214 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.920743 13756 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.924341 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:34.925287 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:34.926025 12068 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:34.853551 13716 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:34.926971 11484 connection.cc:256] RPC call timeout handler was delayed by -1.0378272099999322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29300000000000004s for a response
I20260709 13:15:34.928110 13667 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:34.928889 13667 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.929190 13667 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:34.929977 13667 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.930590 13681 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.931150 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:34.931514 13667 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.932003 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:34.933496 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:34.934141 11366 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:34.935384 11366 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:34.937608 13482 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:34.936144 11370 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.940955 13790 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:34.938019 13482 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:34.944237 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 672) took 1065 ms. Trace:
I20260709 13:15:34.944034 13482 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:34.864207 12062 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:34.945083 13482 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:34.944679 12063 rpcz_store.cc:276] 0709 13:15:33.878604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.979928 (+101324us) service_pool.cc:224] Handling call
0709 13:15:34.057737 (+ 77809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:34.122993 (+ 65256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.124775 (+  1782us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:34.212536 (+ 87761us) log.cc:844] Serialized 26 byte log entry
0709 13:15:34.391909 (+179373us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:34.394378 (+  2469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.400137 (+  5759us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.943997 (+543860us) raft_consensus.cc:1685] finished
0709 13:15:34.944023 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.944166 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2929,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":135,"thread_start_us":117570,"threads_started":1,"wal-append.queue_time_us":235468}
I20260709 13:15:34.937021 11365 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:34.951315 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:34.953220 13711 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:34.953966 13711 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:34.954425 13711 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:34.849591 13761 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:34.955772 13571 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:34.956229 11362 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:34.957350 13761 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:34.958019 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1767) took 2460 ms. Trace:
I20260709 13:15:34.958429 11515 rpcz_store.cc:276] 0709 13:15:32.497287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.498593 (+  1306us) service_pool.cc:224] Handling call
0709 13:15:33.683092 (+1184499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.693951 (+ 10859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.957808 (+1263857us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540031ef50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:34.957960 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2723492736}
I20260709 13:15:34.960517 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:34.921211 11438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:15:34.963008 11438 raft_consensus.cc:606] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:34.963420 11438 raft_consensus.cc:614] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:34.915341 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:34.968211 11437 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:34.957885 13571 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:15:34.970618 13729 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:34.976191 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:34.978921 11439 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:34.944831 13833 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:34.978883 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 158259us
I20260709 13:15:34.980392 11370 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:34.900713 13664 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:34.981325 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:35.003139 11439 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:34.979738 13776 consensus_meta.cc:287] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.952s	user 0.009s	sys 0.006s
W20260709 13:15:34.980851 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 91742.7us
I20260709 13:15:34.998924 11517 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:15:34.986853 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:35.001309 11513 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:34.997061 11515 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:15:35.007905 11333 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:34.995348 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:35.011811 11518 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:35.001960 11438 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:15:34.989531 13348 consensus_meta.cc:287] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.905s	user 0.009s	sys 0.007s
I20260709 13:15:35.016614 11437 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:34.981801 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:34.931109 13681 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.023067 12034 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:35.018298 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 671) took 1174 ms. Trace:
I20260709 13:15:35.021888 11439 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:35.011133 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:34.997925 13571 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:34.983937 13833 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:35.024209 12034 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:35.025378 12067 rpcz_store.cc:276] 0709 13:15:33.843422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.869869 (+ 26447us) service_pool.cc:224] Handling call
0709 13:15:33.870077 (+   208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.870224 (+   147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.870238 (+    14us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.875213 (+  4975us) log.cc:844] Serialized 26 byte log entry
0709 13:15:34.197430 (+322217us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:34.197789 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.197858 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:35.018100 (+820242us) raft_consensus.cc:1685] finished
0709 13:15:35.018119 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:35.018243 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":261607,"threads_started":1,"wal-append.queue_time_us":494080}
I20260709 13:15:35.017688 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:35.026328 11437 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:35.039197 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.013213 13715 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:35.011212 11512 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:34.994503 13664 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:34.996258 13654 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:34.955304 13711 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.009424 11510 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:35.050081 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:35.040208 13661 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:35.053645 11368 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:15:35.016881 13647 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.008402 13454 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:35.054872 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:35.018324 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:35.053467 11437 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.078693 11437 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:35.049510 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:35.042048 13805 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:35.023099 13845 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:34.999272 13826 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.057324 13634 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:35.061959 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:35.088214 12067 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:35.046339 13790 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.046134 13649 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.050472 13833 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.010254 13776 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.052827 13545 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:35.041702 13571 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.056320 13664 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:35.032230 13348 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.061944 13661 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.060773 13631 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:35.084594 13847 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:35.093601 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:35.098016 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.080428 13805 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:35.165544 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:35.208866 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.168511 11438 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:35.126628 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:35.123325 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.170878 12067 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:35.177227 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.085754 13848 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.186261 11437 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:15:35.092654 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 108022us
I20260709 13:15:35.087760 13634 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.208701 13847 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.207590 11444 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:15:35.060897 13846 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 13:15:35.087476 13656 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 13:15:35.093321 13769 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.153s
I20260709 13:15:35.091361 13472 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:35.080962 13647 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.088132 13716 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:35.099224 13790 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:35.301528 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 206464us
I20260709 13:15:35.411541 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:35.412048 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:35.177155 13571 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:35.302034 11518 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:35.248199 11440 raft_consensus.cc:2368] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6.
I20260709 13:15:35.250586 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:35.436735 11517 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:35.298861 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.252595 11515 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:15:35.093254 13845 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:35.439074 11513 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:35.169086 13851 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:35.442874 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:35.443632 12067 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:35.444382 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:35.444903 12067 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:35.445700 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:35.446420 12067 raft_consensus.cc:2410] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:35.447492 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:35.255738 13829 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:15:35.178344 13661 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:35.208727 13805 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:35.178359 11361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:35.409950 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:35.172946 13649 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.403596 13482 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:15:35.174795 13759 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.280s
I20260709 13:15:35.492055 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:35.494156 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 677) took 1161 ms. Trace:
I20260709 13:15:35.419728 13454 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.420421 13634 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.441840 11512 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:35.438988 13716 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:35.438412 13545 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:35.531461 12067 rpcz_store.cc:276] 0709 13:15:34.332668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.493336 (+1160668us) service_pool.cc:224] Handling call
0709 13:15:35.493651 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:35.493841 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.493866 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:35.493907 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:35.493937 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.493986 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:35.494097 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:35.539280 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 126597us
I20260709 13:15:35.507819 13855 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.450361 13851 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.504796 13631 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:35.510495 13348 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:35.531762 11361 raft_consensus.cc:606] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:35.576508 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:35.529843 12062 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:35.579833 13855 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.580510 13845 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.581013 13741 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:35.582288 13855 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:35.581975 13631 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.583472 13826 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:35.422049 13848 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.584061 12062 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:35.585776 13848 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:35.453084 13854 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.553551 13472 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:15:35.605947 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 157003us
I20260709 13:15:35.579613 11361 raft_consensus.cc:614] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:35.538470 13454 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:35.615799 13856 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:35.616330 13863 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:35.623931 13847 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:35.624396 13849 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 13:15:35.626924 13832 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.035s	user 0.000s	sys 0.002s
I20260709 13:15:35.653280 13454 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:35.654713 13853 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:15:35.635268 13805 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.658205 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:35.658876 13805 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:35.659511 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:35.660683 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:35.661724 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:35.662717 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:35.613786 13776 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:35.631886 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2069) took 1861 ms. Trace:
I20260709 13:15:35.634143 13634 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.611864 13854 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.663851 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2077) took 1058 ms. Trace:
I20260709 13:15:35.673151 11364 rpcz_store.cc:276] 0709 13:15:34.605843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.605985 (+   142us) service_pool.cc:224] Handling call
0709 13:15:34.637157 (+ 31172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:34.637504 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.637533 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:34.645154 (+  7621us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.040963 (+395809us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:35.041010 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.042304 (+  1294us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:35.663603 (+621299us) raft_consensus.cc:1685] finished
0709 13:15:35.663637 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:35.663788 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6736,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":304148,"threads_started":1,"wal-append.queue_time_us":519075}
I20260709 13:15:35.680914 11443 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:35.639894 13790 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:35.575994 13860 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:15:35.641417 13571 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.642220 13858 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:15:35.690037 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 181217us
I20260709 13:15:35.644124 13647 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:35.553640 13716 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.578157 13851 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:35.682988 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.690868 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:35.692320 11516 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:15:35.663367 11365 raft_consensus.cc:2368] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:35.699287 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.703830 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.691776 11518 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:35.639132 13656 consensus_queue.cc:1243] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:15:35.682928 13833 consensus_meta.cc:287] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.509s	user 0.004s	sys 0.004s
I20260709 13:15:35.707337 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:35.664813 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:35.666780 13854 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:35.665966 13634 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:35.709982 11364 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 2.
I20260709 13:15:35.695663 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:35.708122 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.686976 13857 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.714818 11514 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:35.715490 11516 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:35.664237 13661 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.695204 13472 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:35.684273 13764 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:15:35.736022 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:35.737607 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:35.738865 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:35.723757 13797 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.788s	user 0.005s	sys 0.000s
I20260709 13:15:35.739470 11437 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:35.741742 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.742321 11515 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:35.743326 12066 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:35.743593 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.744241 11518 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:35.745048 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:35.745536 11518 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.728137 13571 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:35.695652 13657 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:15:35.751924 11437 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:35.754851 13862 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:35.755263 13862 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:35.755949 13864 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:35.756320 13864 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:35.729697 13664 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.757071 13864 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.758334 13864 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.759059 13664 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:35.705921 13868 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:15:35.664008 11362 rpcz_store.cc:276] 0709 13:15:33.770230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.422188 (+651958us) service_pool.cc:224] Handling call
0709 13:15:34.422479 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:34.956689 (+534210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.631638 (+674949us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54001e5ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:35.631823 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1227079552}
I20260709 13:15:35.760182 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:35.760968 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:35.663913 13545 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.761337 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:35.761886 11367 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:35.732271 13667 consensus_meta.cc:287] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.800s	user 0.007s	sys 0.003s
I20260709 13:15:35.722046 13863 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:35.764535 13863 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:35.764974 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 413) took 1770 ms. Trace:
I20260709 13:15:35.765307 13863 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.766553 13863 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:35.718034 13647 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:15:35.733276 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:35.768344 11368 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2034) took 4772 ms (client timeout 1939 ms). Trace:
W20260709 13:15:35.768720 11368 rpcz_store.cc:269] 0709 13:15:30.996529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.997890 (+  1361us) service_pool.cc:224] Handling call
0709 13:15:35.768139 (+4770249us) spinlock_profiling.cc:243] Waited 4.01 s on lock 0x7b54003228d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:35.768252 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9226863360}
I20260709 13:15:35.765313 11444 rpcz_store.cc:276] 0709 13:15:33.994391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.994537 (+   146us) service_pool.cc:224] Handling call
0709 13:15:33.994855 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.418944 (+1424089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.764753 (+345809us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400386654. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:35.764925 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2788392192}
I20260709 13:15:35.769918 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:35.721553 13739 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:15:35.713435 13833 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.734634 13472 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:35.746631 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:35.746837 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.749059 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 659) took 2106 ms. Trace:
I20260709 13:15:35.725435 13857 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:35.749742 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:35.750368 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:35.751140 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:35.787700 11513 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:35.764020 13764 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:35.760998 13841 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.816s	user 0.001s	sys 0.004s
I20260709 13:15:35.774173 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 948) took 1160 ms. Trace:
I20260709 13:15:35.774806 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.749004 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.751464 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:35.748603 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.749366 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.781805 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:35.733834 13661 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:35.768800 13716 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:35.871199 11513 raft_consensus.cc:2410] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:37.047685 12063 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:36.133860 13781 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:36.152040 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 467199us
I20260709 13:15:35.988723 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:37.050318 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1909) took 1107 ms. Trace:
I20260709 13:15:36.391986 13833 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:36.074066 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.391317 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:35.860852 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:36.381244 11369 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 498) took 2621 ms (client timeout 1902 ms). Trace:
W20260709 13:15:35.969952 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.003314 11518 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:15:36.380110 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.064946 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.852149 11442 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:35.795621 11441 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:36.012377 13482 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:36.457159 13472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.008s	sys 0.005s
I20260709 13:15:35.932875 11511 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:35.865125 11367 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:37.072325 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2080) took 1318 ms. Trace:
W20260709 13:15:36.112294 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 43626.5us
I20260709 13:15:37.072718 11367 rpcz_store.cc:276] 0709 13:15:35.754009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.761089 (+  7080us) service_pool.cc:224] Handling call
0709 13:15:37.072263 (+1311174us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:36.065599 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2179) took 1330 ms (client timeout 1525 ms). Trace:
I20260709 13:15:37.045352 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:35.848573 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 415) took 1014 ms. Trace:
I20260709 13:15:37.055282 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 515) took 1312 ms. Trace:
I20260709 13:15:36.007256 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.048990 13781 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:37.052114 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.018450 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:37.075217 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1554) took 1976 ms. Trace:
I20260709 13:15:37.058545 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1907) took 1243 ms. Trace:
I20260709 13:15:37.054148 11519 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
W20260709 13:15:37.073647 11443 rpcz_store.cc:269] 0709 13:15:34.735701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.736301 (+   600us) service_pool.cc:224] Handling call
0709 13:15:36.065412 (+1329111us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b5400398854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:36.065535 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1974713088}
I20260709 13:15:37.075709 11442 rpcz_store.cc:276] 0709 13:15:35.815687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.815860 (+   173us) service_pool.cc:224] Handling call
0709 13:15:35.851965 (+ 36105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.058312 (+1206347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.058492 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.074270 11517 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.076000 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 513) took 1333 ms. Trace:
W20260709 13:15:37.059315 12063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 515) took 2227 ms (client timeout 2850 ms). Trace:
W20260709 13:15:37.082481 11516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 509) took 1341 ms (client timeout 1617 ms). Trace:
W20260709 13:15:37.082918 12068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 510) took 2291 ms (client timeout 2386 ms). Trace:
W20260709 13:15:36.456960 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.083531 12068 rpcz_store.cc:269] 0709 13:15:34.791868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.386974 (+1595106us) service_pool.cc:224] Handling call
0709 13:15:37.082848 (+695874us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.083972 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.085637 11335 leader_election.cc:336] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.537s (SENT)
I20260709 13:15:37.086071 11335 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:37.053797 11445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.000s	sys 0.013s
W20260709 13:15:37.046458 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 1.59897e+06us
W20260709 13:15:37.086858 11445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.000s	sys 0.013s
I20260709 13:15:37.053505 11444 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:37.048379 11514 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:37.059964 11436 raft_consensus.cc:2376] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8. Re-sending same reply.
I20260709 13:15:37.047938 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 508) took 1333 ms. Trace:
I20260709 13:15:37.056095 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1906) took 1269 ms. Trace:
I20260709 13:15:37.049201 12067 rpcz_store.cc:276] 0709 13:15:34.614243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.551349 (+937106us) service_pool.cc:224] Handling call
0709 13:15:35.551690 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.552079 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:35.575959 (+ 23880us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.578317 (+  2358us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.717000 (+138683us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:35.717039 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.717099 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:35.773959 (+ 56860us) raft_consensus.cc:1685] finished
0709 13:15:35.773981 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:35.774115 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":15096,"thread_start_us":32530,"threads_started":1,"wal-append.queue_time_us":151064,"wal-append.run_cpu_time_us":1506,"wal-append.run_wall_time_us":1008350}
I20260709 13:15:37.044869 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:37.050669 11439 rpcz_store.cc:276] 0709 13:15:35.942793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.942911 (+   118us) service_pool.cc:224] Handling call
0709 13:15:37.050258 (+1107347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.050854 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:37.070427 11515 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:37.022017 11408 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.020s	sys 0.019s
I20260709 13:15:37.073896 11438 rpcz_store.cc:276] 0709 13:15:34.833688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.022863 (+189175us) service_pool.cc:224] Handling call
0709 13:15:35.049983 (+ 27120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:35.245581 (+195598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.848347 (+602766us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400314cd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:35.848514 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272112896}
I20260709 13:15:37.075279 11437 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:37.089425 11408 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.021s	sys 0.020s
I20260709 13:15:37.076895 11519 rpcz_store.cc:276] 0709 13:15:35.742313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.748164 (+  5851us) service_pool.cc:224] Handling call
0709 13:15:37.075930 (+1327766us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.090045 11515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 514) took 1347 ms (client timeout 1598 ms). Trace:
W20260709 13:15:37.053344 11369 rpcz_store.cc:269] 0709 13:15:33.753350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.874125 (+120775us) service_pool.cc:224] Handling call
0709 13:15:36.374976 (+2500851us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54001e5fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:36.375141 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297793408}
W20260709 13:15:37.090767 11515 rpcz_store.cc:269] 0709 13:15:35.742605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.742965 (+   360us) service_pool.cc:224] Handling call
0709 13:15:37.089989 (+1347024us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.091653 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 1.67762e+06us
I20260709 13:15:37.091768 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.092228 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1905) took 1310 ms. Trace:
I20260709 13:15:37.092514 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2083) took 1337 ms. Trace:
I20260709 13:15:37.092571 11444 rpcz_store.cc:276] 0709 13:15:35.781393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.781550 (+   157us) service_pool.cc:224] Handling call
0709 13:15:37.092169 (+1310619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.092912 11369 rpcz_store.cc:276] 0709 13:15:35.754702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.091530 (+1336828us) service_pool.cc:224] Handling call
0709 13:15:37.092462 (+   932us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:36.865298 13863 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.098s	user 0.006s	sys 0.006s
W20260709 13:15:35.819329 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 348159us
W20260709 13:15:37.089636 11516 rpcz_store.cc:269] 0709 13:15:35.741067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.748369 (+  7302us) service_pool.cc:224] Handling call
0709 13:15:37.082413 (+1334044us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:36.056766 13808 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 2.063s	user 0.004s	sys 0.010s
I20260709 13:15:37.089557 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:37.102813 11368 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:37.087711 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 516) took 1315 ms. Trace:
I20260709 13:15:37.103763 11514 rpcz_store.cc:276] 0709 13:15:35.772633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.774543 (+  1910us) service_pool.cc:224] Handling call
0709 13:15:37.087656 (+1313113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.104676 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.105201 11514 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.095794 11513 rpcz_store.cc:276] 0709 13:15:35.714410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.745564 (+ 31154us) service_pool.cc:224] Handling call
0709 13:15:37.047871 (+1302307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.106006 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.096246 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1908) took 1235 ms. Trace:
I20260709 13:15:37.107096 11436 rpcz_store.cc:276] 0709 13:15:35.860466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.860588 (+   122us) service_pool.cc:224] Handling call
0709 13:15:37.096203 (+1235615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.096432 11441 rpcz_store.cc:276] 0709 13:15:35.786447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.786632 (+   185us) service_pool.cc:224] Handling call
0709 13:15:35.786918 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.055852 (+1268934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.056043 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.077723 11518 rpcz_store.cc:276] 0709 13:15:35.742872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.744837 (+  1965us) service_pool.cc:224] Handling call
0709 13:15:37.055222 (+1310385us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.088685 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.076970 11364 rpcz_store.cc:276] 0709 13:15:35.098628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.745603 (+646975us) service_pool.cc:224] Handling call
0709 13:15:37.075152 (+1329549us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:36.275835 13818 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.352s	user 0.020s	sys 0.013s
W20260709 13:15:36.415472 13650 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.011s	sys 0.007s
I20260709 13:15:37.098337 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:35.786006 13634 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.108385 13857 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:37.052964 13844 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.032s	user 0.001s	sys 0.014s
W20260709 13:15:37.137452 13844 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.002s	sys 0.014s
I20260709 13:15:37.138038 13844 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:37.138438 13844 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:37.138803 13844 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:37.046372 13644 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.007s	sys 0.031s
I20260709 13:15:37.100040 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:37.150648 12067 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:37.093986 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.151427 11367 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:15:37.151885 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1555) took 1989 ms. Trace:
I20260709 13:15:37.152195 11367 rpcz_store.cc:276] 0709 13:15:35.162346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.093768 (+1931422us) service_pool.cc:224] Handling call
0709 13:15:37.151821 (+ 58053us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.097276 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46494.4us
I20260709 13:15:37.153448 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:37.154743 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:37.108438 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.159991 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:37.160952 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 519) took 1316 ms. Trace:
I20260709 13:15:37.161259 11513 rpcz_store.cc:276] 0709 13:15:35.845037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.106937 (+1261900us) service_pool.cc:224] Handling call
0709 13:15:37.160886 (+ 53949us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.091672 13833 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:37.056310 13482 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:35.845968 13790 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.117360 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:37.176658 12065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 516) took 2316 ms (client timeout 2431 ms). Trace:
W20260709 13:15:37.177052 12065 rpcz_store.cc:269] 0709 13:15:34.860112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.071349 (+2211237us) service_pool.cc:224] Handling call
0709 13:15:37.176579 (+105230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.117942 11516 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:37.178052 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 523) took 1024 ms. Trace:
I20260709 13:15:37.178395 11516 rpcz_store.cc:276] 0709 13:15:36.153221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.098123 (+944902us) service_pool.cc:224] Handling call
0709 13:15:37.177985 (+ 79862us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.179308 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.179947 11516 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:37.118783 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:36.724690 13619 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:15:37.071776 13472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.010s	sys 0.015s
I20260709 13:15:37.098887 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1910) took 1124 ms. Trace:
I20260709 13:15:37.182792 11437 rpcz_store.cc:276] 0709 13:15:35.975078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.975775 (+   697us) service_pool.cc:224] Handling call
0709 13:15:37.098836 (+1123061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.052196 13662 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:37.148829 13644 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.008s	sys 0.031s
I20260709 13:15:35.804059 13764 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:36.165534 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.126639 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:37.115588 11517 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:36.134248 13681 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:36.001523 13846 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:37.082922 12063 rpcz_store.cc:269] 0709 13:15:34.832227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.751152 (+918925us) service_pool.cc:224] Handling call
0709 13:15:37.059257 (+1308105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.132652 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1564) took 1468 ms. Trace:
W20260709 13:15:37.133476 13877 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.325s	user 0.000s	sys 0.000s
I20260709 13:15:37.197665 11516 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
W20260709 13:15:37.116927 13818 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.193s	user 0.027s	sys 0.019s
W20260709 13:15:37.117254 13650 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.614s	user 0.020s	sys 0.032s
I20260709 13:15:37.136072 13634 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.083179 13781 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.001350 13851 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:37.166812 11364 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:37.232643 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1551) took 2149 ms. Trace:
I20260709 13:15:37.233024 11364 rpcz_store.cc:276] 0709 13:15:35.083490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.123458 (+2039968us) service_pool.cc:224] Handling call
0709 13:15:37.166628 (+ 43170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.232336 (+ 65708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.232575 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.169749 12067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 958) took 1555 ms (client timeout 1943 ms). Trace:
W20260709 13:15:37.234601 12067 rpcz_store.cc:269] 0709 13:15:35.614215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.099788 (+1485573us) service_pool.cc:224] Handling call
0709 13:15:37.169676 (+ 69888us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.174192 11518 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:37.236351 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:37.235664 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:37.237157 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 517) took 1464 ms. Trace:
I20260709 13:15:37.183967 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1912) took 1057 ms. Trace:
I20260709 13:15:37.129370 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.237768 11442 rpcz_store.cc:276] 0709 13:15:36.126684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.076857 (+950173us) service_pool.cc:224] Handling call
0709 13:15:37.183895 (+107038us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.239090 11518 rpcz_store.cc:276] 0709 13:15:35.773019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.236134 (+1463115us) service_pool.cc:224] Handling call
0709 13:15:37.237096 (+   962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.239920 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:37.240854 11518 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:37.241209 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 518) took 1396 ms. Trace:
I20260709 13:15:37.242237 11437 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:37.241523 11518 rpcz_store.cc:276] 0709 13:15:35.844703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.239698 (+1394995us) service_pool.cc:224] Handling call
0709 13:15:37.241159 (+  1461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.243386 11518 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:37.243928 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 521) took 1254 ms. Trace:
I20260709 13:15:37.244249 11518 rpcz_store.cc:276] 0709 13:15:35.989477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.242972 (+1253495us) service_pool.cc:224] Handling call
0709 13:15:37.243229 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.243727 (+   498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.243881 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.222488 13740 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:37.248036 11364 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:37.185175 13711 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:37.188200 11517 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 520) took 1342 ms (client timeout 1615 ms). Trace:
I20260709 13:15:37.163918 13857 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.120096 13661 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:15:37.133985 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:37.184594 13790 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.192723 11367 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:37.257019 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1559) took 1819 ms. Trace:
I20260709 13:15:37.222939 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:37.193199 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.257400 11367 rpcz_store.cc:276] 0709 13:15:35.437861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.170182 (+1732321us) service_pool.cc:224] Handling call
0709 13:15:37.192538 (+ 22356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.256741 (+ 64203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.256957 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.182510 13619 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:37.225342 11368 rpcz_store.cc:276] 0709 13:15:35.663919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.044856 (+1380937us) service_pool.cc:224] Handling call
0709 13:15:37.132565 (+ 87709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.226004 11514 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:37.261310 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:37.262128 11514 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:37.180989 11441 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:37.237224 12067 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.181927 11369 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:37.266418 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1549) took 2183 ms. Trace:
I20260709 13:15:37.266785 11369 rpcz_store.cc:276] 0709 13:15:35.083032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.170566 (+2087534us) service_pool.cc:224] Handling call
0709 13:15:37.181747 (+ 11181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.266143 (+ 84396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.266358 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.251430 11517 rpcz_store.cc:269] 0709 13:15:35.854805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.006992 (+152187us) service_pool.cc:224] Handling call
0709 13:15:37.188131 (+1181139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.252951 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.183355 13833 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.253448 12068 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.222123 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 522) took 1107 ms. Trace:
I20260709 13:15:37.269529 11519 rpcz_store.cc:276] 0709 13:15:36.114407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.090821 (+976414us) service_pool.cc:224] Handling call
0709 13:15:37.221967 (+131146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.254845 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1560) took 1747 ms. Trace:
I20260709 13:15:37.271184 11364 rpcz_store.cc:276] 0709 13:15:35.507706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.234135 (+1726429us) service_pool.cc:224] Handling call
0709 13:15:37.247845 (+ 13710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.250596 (+  2751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.254798 (+  4202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.244196 13864 consensus_meta.cc:287] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.485s	user 0.004s	sys 0.009s
I20260709 13:15:37.252418 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:37.249257 11362 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:37.273442 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:37.281678 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 674) took 3248 ms. Trace:
I20260709 13:15:37.279636 13899 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:37.297640 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:37.252565 13855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.221s	user 0.060s	sys 0.100s
W20260709 13:15:37.256637 13839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.329s	user 0.053s	sys 0.112s
I20260709 13:15:37.282178 12069 rpcz_store.cc:276] 0709 13:15:34.032856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.417145 (+384289us) service_pool.cc:224] Handling call
0709 13:15:34.457219 (+ 40074us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:34.484948 (+ 27729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.484980 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:34.485538 (+   558us) log.cc:844] Serialized 52 byte log entry
0709 13:15:34.651293 (+165755us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:34.658678 (+  7385us) log.cc:844] Serialized 14 byte log entry
0709 13:15:34.792352 (+133674us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b18003d0480. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:34.802656 (+ 10304us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b18003d0480. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:15:34.995753 (+193097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.995832 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:35.589069 (+593237us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b48001b0e58. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f 00007f35c1a15f37 00007f35c1a2a55c 00007f35c1a2a508 00007f35c1a2a488 00007f35c1a2a443 00007f35c1a296ec 00007f35c0953a96
0709 13:15:35.760451 (+171382us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:37.281395 (+1520944us) raft_consensus.cc:1685] finished
0709 13:15:37.281421 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.281605 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27321,"prepare.queue_time_us":7115,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":1018,"raft.queue_time_us":346748,"raft.run_cpu_time_us":2599,"raft.run_wall_time_us":862356,"spinlock_wait_cycles":153116160,"thread_start_us":259457,"threads_started":2,"wal-append.queue_time_us":389893}
W20260709 13:15:37.287010 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 164465us
I20260709 13:15:37.287774 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:37.191859 13662 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.250385 13711 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:37.223757 13846 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.299129 11518 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:15:37.301079 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:37.302174 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 216579us
I20260709 13:15:37.303514 11441 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:37.311980 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 267190us
I20260709 13:15:37.313555 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2082) took 1559 ms. Trace:
I20260709 13:15:37.345552 11367 rpcz_store.cc:276] 0709 13:15:35.754465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.264184 (+1509719us) service_pool.cc:224] Handling call
0709 13:15:37.313014 (+ 48830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.313207 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:37.313232 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:37.313273 (+    41us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:37.313302 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.313354 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.313485 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.315801 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 242603us
I20260709 13:15:37.258625 13857 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:37.330180 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:37.310635 13855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.285s	user 0.061s	sys 0.100s
W20260709 13:15:37.336244 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 181552us
I20260709 13:15:37.333519 13644 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:15:37.317917 13839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.393s	user 0.053s	sys 0.112s
I20260709 13:15:37.328491 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:37.380589 11437 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:37.381460 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:37.382663 11437 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:37.382956 11437 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:37.340065 12068 raft_consensus.cc:2410] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:15:37.344949 12032 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:37.327975 13899 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:37.363270 12069 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:37.364562 13833 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:37.365548 13903 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:37.253822 13764 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.318784 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2081) took 1564 ms. Trace:
I20260709 13:15:37.368820 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:37.371146 13644 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:37.402186 11436 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:37.341038 13711 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.371614 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:37.372233 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:37.368320 13902 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:15:37.369181 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:37.407255 11441 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:37.401224 11362 rpcz_store.cc:276] 0709 13:15:35.754248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.165167 (+410919us) service_pool.cc:224] Handling call
0709 13:15:37.315314 (+1150147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.401605 13903 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.408497 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:37.409920 11362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:37.410504 11362 raft_consensus.cc:606] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:37.410892 11362 raft_consensus.cc:614] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:37.414309 11362 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:15:37.415823 13898 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.359s	user 0.002s	sys 0.000s
I20260709 13:15:37.416074 13626 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.416797 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 512) took 2422 ms. Trace:
I20260709 13:15:35.786631 12064 rpcz_store.cc:276] 0709 13:15:33.642270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.648433 (+  6163us) service_pool.cc:224] Handling call
0709 13:15:33.667742 (+ 19309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.714603 (+ 46861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.714642 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.715190 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.822432 (+107242us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:33.828870 (+  6438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.828948 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:35.723169 (+1894221us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:35.742706 (+ 19537us) raft_consensus.cc:1685] finished
0709 13:15:35.742726 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:35.748990 (+  6264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28183,"raft.queue_time_us":26665,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":27633,"spinlock_wait_cycles":21438208,"thread_start_us":93225,"threads_started":1,"wal-append.queue_time_us":150556,"wal-append.run_cpu_time_us":6851,"wal-append.run_wall_time_us":2871834}
I20260709 13:15:37.417198 11370 rpcz_store.cc:276] 0709 13:15:34.994837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.994956 (+   119us) service_pool.cc:224] Handling call
0709 13:15:35.772361 (+777405us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:35.772544 (+   183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.772881 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.772909 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.773349 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:15:37.056683 (+1283334us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:37.099255 (+ 42572us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.294069 (+194814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.294161 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.401708 (+107547us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:37.416596 (+ 14888us) raft_consensus.cc:1685] finished
0709 13:15:37.416619 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.416743 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2300,"prepare.queue_time_us":80077,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":136335,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":60741,"spinlock_wait_cycles":75902208,"thread_start_us":508762,"threads_started":2,"wal-append.queue_time_us":637125}
I20260709 13:15:37.418807 13857 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:37.419487 13851 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:37.420603 13886 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 13:15:37.422089 13855 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:15:37.378146 13888 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.320s	user 0.001s	sys 0.001s
I20260709 13:15:37.384588 12032 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:37.348517 12067 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:15:37.430677 12067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 681) took 1649 ms (client timeout 1838 ms). Trace:
W20260709 13:15:37.431104 12067 rpcz_store.cc:269] 0709 13:15:35.781026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.235310 (+1454284us) service_pool.cc:224] Handling call
0709 13:15:37.430614 (+195304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.385797 13634 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:37.391433 12068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 955) took 1778 ms (client timeout 1731 ms). Trace:
W20260709 13:15:37.478101 12068 rpcz_store.cc:269] 0709 13:15:35.613183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.097212 (+1484029us) service_pool.cc:224] Handling call
0709 13:15:37.391316 (+294104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.479038 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:37.479885 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 959) took 1865 ms. Trace:
I20260709 13:15:37.480216 12068 rpcz_store.cc:276] 0709 13:15:35.614649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.478772 (+1864123us) service_pool.cc:224] Handling call
0709 13:15:37.479826 (+  1054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.391296 11437 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.393536 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 957) took 1779 ms. Trace:
I20260709 13:15:37.395167 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:37.481508 12069 rpcz_store.cc:276] 0709 13:15:35.613782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.329902 (+1716120us) service_pool.cc:224] Handling call
0709 13:15:37.393463 (+ 63561us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.395957 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:37.482506 11437 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.482594 11369 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.398481 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:37.399077 13885 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:15:37.399680 11367 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:37.381376 13907 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.392513 12063 raft_consensus.cc:2410] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:37.404543 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:37.404538 11438 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:37.404986 11439 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:37.410611 13903 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:37.379258 13887 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:15:37.403080 13912 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:37.430610 13661 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.463460 13844 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.361984 13619 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.466069 13649 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.467034 13756 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.467952 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:37.414444 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:15:37.471822 11440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2160) took 3046 ms (client timeout 2652 ms). Trace:
W20260709 13:15:37.510397 11440 rpcz_store.cc:269] 0709 13:15:34.425291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.425403 (+   112us) service_pool.cc:224] Handling call
0709 13:15:37.471604 (+3046201us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54002ea754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:37.471748 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718561664}
W20260709 13:15:37.474522 12066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 947) took 2860 ms (client timeout 3099 ms). Trace:
W20260709 13:15:37.511230 12066 rpcz_store.cc:269] 0709 13:15:34.613998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.016566 (+402568us) service_pool.cc:224] Handling call
0709 13:15:35.491894 (+475328us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:37.474314 (+1982420us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400595bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:37.474436 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490320384}
I20260709 13:15:37.512545 12066 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:37.513160 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 521) took 2436 ms. Trace:
I20260709 13:15:37.513432 12066 rpcz_store.cc:276] 0709 13:15:35.077153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.512021 (+2434868us) service_pool.cc:224] Handling call
0709 13:15:37.512344 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:37.512902 (+   558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.513102 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.514650 12066 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:15:37.475346 11366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1548) took 2392 ms (client timeout 1625 ms). Trace:
I20260709 13:15:37.477159 13790 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:37.485251 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 100714us
W20260709 13:15:37.515091 11366 rpcz_store.cc:269] 0709 13:15:35.082721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.083667 (+   946us) service_pool.cc:224] Handling call
0709 13:15:37.475136 (+2391469us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54002db254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:37.475274 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686387072}
I20260709 13:15:37.515996 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:37.516342 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 522) took 2420 ms. Trace:
I20260709 13:15:37.516778 12066 rpcz_store.cc:276] 0709 13:15:35.095520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.514174 (+2418654us) service_pool.cc:224] Handling call
0709 13:15:37.514452 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:37.516083 (+  1631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.516275 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.517136 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:37.518154 12066 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:15:37.518183 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:37.487471 12069 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:15:37.518944 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 524) took 2219 ms. Trace:
I20260709 13:15:37.519217 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 519) took 2505 ms. Trace:
I20260709 13:15:37.519330 12066 rpcz_store.cc:276] 0709 13:15:35.299502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.517599 (+2218097us) service_pool.cc:224] Handling call
0709 13:15:37.517846 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.518676 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.518895 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.521977 12065 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:37.493090 11437 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:15:37.498466 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:37.522866 12068 raft_consensus.cc:2410] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:37.523281 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 962) took 1893 ms. Trace:
I20260709 13:15:37.523614 12068 rpcz_store.cc:276] 0709 13:15:35.630340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.480811 (+1850471us) service_pool.cc:224] Handling call
0709 13:15:37.523225 (+ 42414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.487988 13839 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:37.503436 12063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 956) took 1888 ms (client timeout 2282 ms). Trace:
I20260709 13:15:37.496027 13907 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.511704 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:37.532440 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:37.524711 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.524514 11437 pending_rounds.cc:85] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 5
I20260709 13:15:37.422201 13911 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:37.519637 12069 rpcz_store.cc:276] 0709 13:15:35.013906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.486852 (+2472946us) service_pool.cc:224] Handling call
0709 13:15:37.487220 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:37.518989 (+ 31769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.519168 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.530643 13912 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:37.537278 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:37.536597 12069 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:37.538949 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 171959us
I20260709 13:15:37.509334 11369 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:37.539088 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 680) took 1758 ms. Trace:
W20260709 13:15:37.532004 12060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 511) took 2739 ms (client timeout 2575 ms). Trace:
I20260709 13:15:37.476323 13644 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.528270 13649 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:37.539676 13839 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:37.503105 13661 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.531020 13790 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:37.544193 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:37.532446 12063 rpcz_store.cc:269] 0709 13:15:35.613510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.255283 (+1641773us) service_pool.cc:224] Handling call
0709 13:15:37.502347 (+247064us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.546718 12069 rpcz_store.cc:276] 0709 13:15:35.780771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.536121 (+1755350us) service_pool.cc:224] Handling call
0709 13:15:37.536423 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.538857 (+  2434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.539036 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.537032 11437 pending_rounds.cc:107] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:15:37.529753 13619 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.548059 13899 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:37.538308 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 961) took 1908 ms. Trace:
W20260709 13:15:37.545207 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 201449us
I20260709 13:15:37.601657 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.555459 13910 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:15:37.605135 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.605803 11367 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:37.606616 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.607465 11370 raft_consensus.cc:2410] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:37.612133 12068 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:37.613842 12069 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:37.614490 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 682) took 1811 ms. Trace:
I20260709 13:15:37.614779 12069 rpcz_store.cc:276] 0709 13:15:35.803627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.597491 (+1793864us) service_pool.cc:224] Handling call
0709 13:15:37.613375 (+ 15884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.614248 (+   873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.614435 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.616235 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 683) took 1812 ms. Trace:
I20260709 13:15:37.617163 13899 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:37.556913 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 188543us
I20260709 13:15:37.618677 13907 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:37.545083 13903 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.619697 13907 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:37.604036 13644 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:37.624619 13911 raft_consensus.cc:1064] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:15:37.601193 13649 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:37.627151 13911 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:37.580387 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.602222 13634 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:37.548840 12060 rpcz_store.cc:269] 0709 13:15:34.792135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.527294 (+2735159us) service_pool.cc:211] Skipping call since client already timed out
0709 13:15:37.530904 (+  3610us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:15:37.604426 12064 rpcz_store.cc:276] 0709 13:15:35.629655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.467661 (+1838006us) service_pool.cc:224] Handling call
0709 13:15:37.535846 (+ 68185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.630347 12064 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:37.558079 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:37.538460 13912 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.632066 11483 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:37.555577 13790 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.598773 13901 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.627740 13649 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.628106 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.596822 11437 raft_consensus.cc:2889] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:37.616701 12069 rpcz_store.cc:276] 0709 13:15:35.803915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.615487 (+1811572us) service_pool.cc:224] Handling call
0709 13:15:37.615765 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.615934 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:37.615958 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:37.615998 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:37.616029 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.616076 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.616183 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:37.636406 11335 leader_election.cc:336] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.925s (SENT)
W20260709 13:15:37.633522 13788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.692s	user 0.022s	sys 0.009s
W20260709 13:15:37.651226 11335 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.926s (SENT)
I20260709 13:15:37.627300 13844 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.638337 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:37.646665 13914 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:37.647347 13903 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:37.652094 11335 leader_election.cc:336] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 2.068s (SENT)
I20260709 13:15:37.662488 13914 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:37.662581 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:37.663636 11335 leader_election.cc:336] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 2.319s (SENT)
I20260709 13:15:37.663470 13914 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.664045 11335 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:37.630968 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 685) took 1753 ms. Trace:
I20260709 13:15:37.664644 12064 rpcz_store.cc:276] 0709 13:15:35.878003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.629906 (+1751903us) service_pool.cc:224] Handling call
0709 13:15:37.630180 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.630711 (+   531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.630899 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:37.622944 13781 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.666586 11444 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:37.668704 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.669564 13900 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.669902 13781 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:37.670475 13739 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:37.671118 13873 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:37.673238 13903 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.638384 13912 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.675808 13873 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:37.676699 13900 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:37.554454 13839 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:37.661705 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:37.638082 13545 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:37.666620 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 68220.8us
W20260709 13:15:37.653208 13788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.711s	user 0.022s	sys 0.009s
I20260709 13:15:37.684033 13741 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:37.684437 13482 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:37.554828 13913 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:37.641831 13790 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.681092 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:37.652678 13649 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:37.687274 13852 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 2.305s	user 0.011s	sys 0.005s
I20260709 13:15:37.667199 13864 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.690074 13917 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:37.638266 13634 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.695652 13594 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.682508 13839 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.707311 13634 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:37.707882 13634 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:37.693058 13711 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.710090 11514 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:37.711040 13711 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:37.703192 11483 connection.cc:256] RPC call timeout handler was delayed by -1.2834247369999048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30990000000000006s for a response
I20260709 13:15:37.706080 13594 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.684471 13741 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:37.715674 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 525) took 2324 ms. Trace:
I20260709 13:15:37.652212 13901 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.682106 13739 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:15:37.682219 13873 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:15:37.705349 13848 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:37.705559 13482 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:15:37.725795 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 167789us
I20260709 13:15:37.692408 13644 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:15:37.688832 13545 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:37.726792 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:37.705696 13864 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.730453 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:37.719223 12066 rpcz_store.cc:276] 0709 13:15:35.391576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.520451 (+2128875us) service_pool.cc:224] Handling call
0709 13:15:37.520718 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:37.529528 (+  8810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:37.529557 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:37.533014 (+  3457us) log.cc:844] Serialized 26 byte log entry
0709 13:15:37.634428 (+101414us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:37.642490 (+  8062us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.715328 (+ 72838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.715445 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.715489 (+    44us) raft_consensus.cc:1685] finished
0709 13:15:37.715502 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.715616 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14584,"prepare.queue_time_us":7944,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":35,"raft.queue_time_us":40995,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":3850,"thread_start_us":148440,"threads_started":2,"wal-append.queue_time_us":139460}
I20260709 13:15:37.692660 13913 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.735881 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 675) took 3667 ms. Trace:
I20260709 13:15:37.724017 13741 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.734663 13844 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:37.728906 13482 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.742753 13901 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.743319 13482 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:37.743654 13844 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:37.744415 13844 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.745638 13790 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.756583 13912 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.758209 13912 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.759445 13903 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:37.759560 13913 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.785648 11363 raft_consensus.cc:2410] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:37.788743 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:37.792446 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:37.792953 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:37.803252 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.731258 12062 raft_consensus.cc:1217] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-5.7]   Dedup: 5.7->[]
W20260709 13:15:37.805768 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 91058.6us
I20260709 13:15:37.806742 12062 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:15:37.807291 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.808288 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:37.808563 11367 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:37.736259 12061 rpcz_store.cc:276] 0709 13:15:34.068634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.470098 (+401464us) service_pool.cc:224] Handling call
0709 13:15:34.581056 (+110958us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:34.825255 (+244199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.838199 (+ 12944us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:34.944072 (+105873us) log.cc:844] Serialized 78 byte log entry
0709 13:15:35.092184 (+148112us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:35.097381 (+  5197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.097477 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.686720 (+2589243us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:37.727515 (+ 40795us) raft_consensus.cc:1685] finished
0709 13:15:37.727541 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.735810 (+  8269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21471,"raft.queue_time_us":149451,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":41930,"spinlock_wait_cycles":5086080,"thread_start_us":284491,"threads_started":2,"wal-append.queue_time_us":312351}
I20260709 13:15:37.811245 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:37.812358 11367 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:37.814687 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:37.816051 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.810967 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:37.829909 13545 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.830659 13482 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:37.830791 13545 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:15:37.821820 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 91338.2us
I20260709 13:15:37.832568 13626 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:37.835894 13707 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:37.836774 13545 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:37.837175 13626 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:37.832779 13644 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:37.839371 13644 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:37.839927 13711 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:37.840669 13711 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:37.841797 13644 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.842832 13711 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.843636 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:37.833369 13482 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:37.831364 11367 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:37.752679 13741 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.857615 13482 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:37.858204 13844 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:37.858669 13844 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.859048 13844 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:37.861855 13900 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:37.753639 13839 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.871873 13482 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:37.868148 13790 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.873864 12060 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:37.874328 13839 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.863739 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:37.852340 11335 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.568s (SENT)
I20260709 13:15:37.883431 13913 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.888693 13913 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.889912 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:37.897253 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:37.908926 12033 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:37.915198 13873 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:37.916690 12065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 954) took 2303 ms (client timeout 2006 ms). Trace:
W20260709 13:15:37.917268 12065 rpcz_store.cc:269] 0709 13:15:35.612859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.177749 (+1564890us) service_pool.cc:224] Handling call
0709 13:15:37.916099 (+738350us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54004b7154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:37.916393 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606186240}
I20260709 13:15:37.920646 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 960) took 2291 ms. Trace:
I20260709 13:15:37.921176 12068 rpcz_store.cc:276] 0709 13:15:35.629372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.524226 (+1894854us) service_pool.cc:224] Handling call
0709 13:15:37.532992 (+  8766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.612541 (+ 79549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.920073 (+307532us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b54005afad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:37.920382 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181536128}
I20260709 13:15:37.924737 13913 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:37.932890 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.932458 13649 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.949316 13594 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:37.955390 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 111968us
I20260709 13:15:37.959511 13839 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:37.960114 13839 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:37.961315 13839 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.961936 13662 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:37.956308 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 102941us
I20260709 13:15:37.964649 12068 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:37.965497 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:37.964485 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 118982us
I20260709 13:15:37.942705 13844 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:37.970413 12067 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:37.979920 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:37.980502 12068 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:37.974505 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:37.983202 11335 leader_election.cc:336] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.696s (SENT)
I20260709 13:15:37.983825 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:37.984668 13594 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:37.985524 11335 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.784s (SENT)
W20260709 13:15:37.985759 13882 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.926s	user 0.000s	sys 0.003s
W20260709 13:15:37.986366 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.987898 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 502) took 2320 ms. Trace:
I20260709 13:15:37.988380 11512 rpcz_store.cc:276] 0709 13:15:35.666988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.667206 (+   218us) service_pool.cc:224] Handling call
0709 13:15:35.667545 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.667862 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:35.667891 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.668358 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:15:35.998570 (+330212us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:35.998624 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.998705 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:37.984756 (+1986051us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:37.987290 (+  2534us) raft_consensus.cc:1685] finished
0709 13:15:37.987317 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.987654 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10119,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":12306,"spinlock_wait_cycles":55260032,"thread_start_us":283515,"threads_started":1,"wal-append.queue_time_us":1390735}
W20260709 13:15:37.990391 13896 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.939s	user 0.002s	sys 0.000s
W20260709 13:15:37.988720 13884 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.930s	user 0.003s	sys 0.000s
I20260709 13:15:37.992022 12065 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:37.991935 13897 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.942s	user 0.000s	sys 0.003s
W20260709 13:15:37.993460 11483 leader_election.cc:336] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.731s (SENT)
I20260709 13:15:37.996009 13905 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:37.996439 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.996987 13905 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:37.996090 11334 connection.cc:256] RPC call timeout handler was delayed by -1.5558269680000194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 13:15:37.999629 11334 connection.cc:256] RPC call timeout handler was delayed by -1.3164269680000871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
I20260709 13:15:38.000065 13905 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:37.996013 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:38.001348 11363 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 495) took 4261 ms (client timeout 1974 ms). Trace:
I20260709 13:15:38.001721 13907 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:15:38.003569 11483 leader_election.cc:336] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.006s (SENT)
I20260709 13:15:38.004057 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.004545 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 425) took 3033 ms. Trace:
I20260709 13:15:38.004884 13907 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:38.006047 11511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 507) took 2307 ms (client timeout 2704 ms). Trace:
W20260709 13:15:38.006742 11483 leader_election.cc:336] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.282s (SENT)
I20260709 13:15:38.007217 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:38.006460 11511 rpcz_store.cc:269] 0709 13:15:35.698835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.699034 (+   199us) service_pool.cc:224] Handling call
0709 13:15:38.005586 (+2306552us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54002c1d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:38.005854 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535192832}
W20260709 13:15:38.001809 11363 rpcz_store.cc:269] 0709 13:15:33.740075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.907912 (+167837us) service_pool.cc:224] Handling call
0709 13:15:38.001031 (+4093119us) spinlock_profiling.cc:243] Waited 3.88 s on lock 0x7b5400422a54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:38.001225 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8912796544}
I20260709 13:15:37.999116 13764 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.004951 11445 rpcz_store.cc:276] 0709 13:15:34.970691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.090232 (+119541us) service_pool.cc:224] Handling call
0709 13:15:35.090466 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:35.090630 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:36.392726 (+1302096us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b1800424840. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35bc5a64bc 00007f35b98f448b 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf
0709 13:15:36.414099 (+ 21373us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.990942 (+1576843us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:38.003589 (+ 12647us) spinlock_profiling.cc:243] Waited 6.47 ms on lock 0x7b1800424840. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:15:38.003886 (+   297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.003975 (+    89us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:38.004091 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.004199 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.004361 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37126,"prepare.queue_time_us":117399,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":20884,"spinlock_wait_cycles":193943552,"thread_start_us":794758,"threads_started":2,"wal-append.queue_time_us":635667}
I20260709 13:15:38.021176 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:38.022456 13781 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:38.030216 13788 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.030491 13781 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.031047 13788 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.033059 13781 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.033587 13781 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.039664 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:38.040079 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 684) took 2216 ms. Trace:
I20260709 13:15:38.040485 12060 rpcz_store.cc:276] 0709 13:15:35.824148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.628795 (+1804647us) service_pool.cc:224] Handling call
0709 13:15:37.629107 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.874379 (+245272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.039778 (+165399us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400599054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:38.040015 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561698688}
I20260709 13:15:38.043346 13833 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:38.044082 11361 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:38.044811 13891 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:38.045243 13891 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:38.046315 13891 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.048286 13950 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.049220 13711 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.048985 13950 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.040445 11367 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:38.051920 13348 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:38.033255 13788 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.052721 13348 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.055745 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.056468 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.057006 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.057318 11515 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:38.057869 11512 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:38.038408 13907 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.055214 13348 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.048503 11369 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [1.3-3.4]
I20260709 13:15:38.051383 13950 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.061347 13348 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.061915 13348 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:38.055823 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.063925 13348 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.064463 13348 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.065006 13348 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.066916 13788 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:38.067673 13818 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.068429 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 518) took 3069 ms. Trace:
I20260709 13:15:38.070361 13864 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.056511 11511 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:15:38.065002 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 686) took 2122 ms. Trace:
I20260709 13:15:38.073071 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 687) took 1933 ms. Trace:
I20260709 13:15:38.083976 13951 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.084791 13951 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.087224 13951 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.088594 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.088392 13862 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.077330 12063 rpcz_store.cc:276] 0709 13:15:35.942150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.659393 (+1717243us) service_pool.cc:224] Handling call
0709 13:15:37.659810 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.806772 (+146962us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:37.807424 (+   652us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.867698 (+ 60274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:37.867791 (+    93us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:37.867850 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.996677 (+128827us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b18001edde0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:38.064600 (+ 67923us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54005212d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:38.064657 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.064813 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13987,"prepare.queue_time_us":39383,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":14628,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":185,"spinlock_wait_cycles":338348672,"thread_start_us":41181,"threads_started":1,"wal-append.queue_time_us":80652}
I20260709 13:15:38.089286 11445 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:38.068315 13950 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:38.059438 13739 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.090905 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:38.095677 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.096171 13862 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.096778 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.071874 13788 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.095153 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.098150 13348 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.103221 11444 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.103758 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:38.087858 13951 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:38.100404 11518 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:15:38.080417 12069 rpcz_store.cc:276] 0709 13:15:36.019064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.924438 (+1905374us) service_pool.cc:224] Handling call
0709 13:15:37.924935 (+   497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.925416 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:37.925828 (+   412us) log.cc:844] Serialized 14 byte log entry
0709 13:15:37.951367 (+ 25539us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:37.951444 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:37.951498 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.951565 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:37.952439 (+   874us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":239,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":60,"raft.queue_time_us":7248,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"spinlock_wait_cycles":230016,"thread_start_us":17153,"threads_started":1,"wal-append.queue_time_us":45530}
I20260709 13:15:38.074937 12062 rpcz_store.cc:276] 0709 13:15:34.999168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.585013 (+585845us) service_pool.cc:224] Handling call
0709 13:15:37.728588 (+2143575us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:37.807430 (+ 78842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.068132 (+260702us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b54001fe850. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:38.068335 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4927219968}
I20260709 13:15:38.103291 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:38.108412 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.109122 11515 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:38.111127 13949 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.112025 13949 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.113799 13949 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.100266 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.115072 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.115664 11409 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.116384 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.117161 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.118875 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.120651 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.121223 11437 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:38.123469 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:38.123618 13950 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:15:38.124243 11370 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:38.127638 11367 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:15:38.117157 11439 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:38.134168 13846 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.136502 13788 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:38.138406 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47866.9us
W20260709 13:15:38.141463 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 48164.4us
I20260709 13:15:38.142378 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:38.126783 11268 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, term changed from 1 to 3, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to <none>, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:38.107043 13649 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:38.152685 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 123179us
I20260709 13:15:38.137259 13348 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.137156 13788 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:38.157101 11440 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.157363 11436 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:38.158409 13661 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.158879 11268 catalog_manager.cc:5697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
W20260709 13:15:38.161660 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.155210 11445 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.166970 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.152616 13594 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:38.169008 13594 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.169425 13594 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:38.145857 13781 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.171705 13844 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.172160 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.172537 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:38.175221 13950 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.134253 13951 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.175894 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:38.143694 13739 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.176712 11365 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:38.177978 13951 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.178599 13739 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.171967 13781 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.179740 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:38.180769 11368 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:15:38.149556 13949 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.187734 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:38.193022 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:38.179363 13740 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:38.194823 13740 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.196210 11334 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.196168 13740 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.199636 13863 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:38.200076 13863 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.200421 13863 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:38.201609 11514 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:38.201911 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:38.193265 13945 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:15:38.148216 13818 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.204151 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:38.206107 13818 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.206241 13864 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:38.206686 13864 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.207068 13864 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.211486 11437 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:38.212772 11436 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:38.202716 11515 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:15:38.194180 13943 log.cc:927] Time spent T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:15:38.219528 11440 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:38.196552 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:38.164790 13348 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.199661 13647 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:38.194823 13942 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:15:38.221482 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:38.223707 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 217032us
I20260709 13:15:38.223958 11442 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:38.224570 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.263865 11436 raft_consensus.cc:2376] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6. Re-sending same reply.
I20260709 13:15:38.266424 11370 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:15:38.223503 11410 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51484 dropped due to backpressure: service queue is full with 50 items
I20260709 13:15:38.227823 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:38.240958 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:38.239678 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 46766.2us
I20260709 13:15:38.242705 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:38.265780 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:38.271224 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.247529 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.269950 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.221347 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.286850 11437 raft_consensus.cc:2376] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5. Re-sending same reply.
I20260709 13:15:38.222385 11268 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:15:38.291788 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:38.221550 11361 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.245126 13891 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.247423 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.301570 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.314390 11436 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:38.223531 13348 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.316869 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.297297 11439 raft_consensus.cc:2376] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5. Re-sending same reply.
I20260709 13:15:38.247921 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:38.227885 13647 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.239928 13662 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:38.289265 11268 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 5, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 5 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:38.221101 13952 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.300256 11370 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:38.307848 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:38.315435 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:38.312212 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.223872 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:38.319677 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:38.311545 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 1.03774e+06us
I20260709 13:15:38.319669 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:38.317544 12069 raft_consensus.cc:2376] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4. Re-sending same reply.
I20260709 13:15:38.323100 11444 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:38.325044 11518 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.241451 13545 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:15:38.363547 13741 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 13:15:38.364423 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1911) took 2168 ms. Trace:
I20260709 13:15:38.365531 12060 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:15:38.367319 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:38.367985 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 127285us
I20260709 13:15:38.239562 13873 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:15:38.324738 13662 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.269999 13839 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:15:38.383950 11440 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:38.381784 12067 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 LEARNER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:38.232365 13421 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
W20260709 13:15:38.390930 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.356961 11365 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:38.398128 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:38.327056 13953 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.268263 11410 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51484 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 441)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2193)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2194)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2184)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 442)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 437)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2187)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2199)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2202)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2185)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2183)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 439)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2188)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 446)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 434)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2182)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 435)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2186)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2196)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2207)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2208)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2209)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2211)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2212)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2214)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2218)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1930)
I20260709 13:15:38.244506 13728 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 13:15:38.379861 11443 rpcz_store.cc:276] 0709 13:15:36.037185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.077310 (+1040125us) service_pool.cc:224] Handling call
0709 13:15:37.093389 (+ 16079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.111290 (+1017901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.111322 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.111364 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:38.111386 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.111483 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.187154 (+ 75671us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b540028a5d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:38.189707 (+  2553us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":918072,"raft.queue_time_us":949068,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":207,"spinlock_wait_cycles":36262144,"thread_start_us":96890,"threads_started":1}
I20260709 13:15:38.326685 13790 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:38.416721 11369 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:38.398650 12063 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:38.383812 11268 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:38.425635 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 139901us
W20260709 13:15:38.426663 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:38.427598 11484 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 2.670s (SENT) [suppressed 54 similar messages]
I20260709 13:15:38.426887 11268 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 3 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:15:38.427503 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 310464us
W20260709 13:15:38.428815 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.430862 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.430936 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:38.439137 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 190244us
I20260709 13:15:38.393137 13662 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:38.461623 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 156360us
I20260709 13:15:38.411448 13846 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.429693 13790 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.466650 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.382436 13864 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.496529 12065 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:38.502051 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.494755 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.416666 11512 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:38.512058 11364 raft_consensus.cc:2410] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:15:38.512323 13958 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:15:38.512458 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:38.513057 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.475592 13953 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.517447 13953 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.521180 11444 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:38.468758 13711 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:38.474929 13790 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.506356 13873 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:38.550892 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.566238 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.507474 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:38.574185 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 145962us
I20260709 13:15:38.501379 13863 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.584405 11514 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:38.596681 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 45877.3us
I20260709 13:15:38.597661 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.591013 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.598270 11443 raft_consensus.cc:2376] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7. Re-sending same reply.
I20260709 13:15:38.559811 13873 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.620370 11444 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
W20260709 13:15:38.627048 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 199474us
I20260709 13:15:38.627734 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.605266 13846 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:38.631052 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 46119.4us
I20260709 13:15:38.628438 11443 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:38.625387 11268 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 6, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 6 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:15:38.598672 13953 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.640097 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:38.641819 11515 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:15:38.643232 11445 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:38.644170 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.644591 11445 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
W20260709 13:15:38.641692 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 174595us
I20260709 13:15:38.648900 11519 raft_consensus.cc:2410] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:38.584415 13956 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:15:38.649542 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.650882 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.635558 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:38.653148 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.648703 12060 raft_consensus.cc:2410] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:15:38.656283 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:38.656502 13916 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:15:38.657096 12065 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:38.658385 13863 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.659950 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.661034 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:38.662120 13626 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:38.647758 13952 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.665503 11438 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:38.650651 13818 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.671062 11517 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.671469 11514 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:38.671823 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.672616 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:38.673697 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.650405 12061 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:38.676185 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.577509 13647 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:38.674619 13594 consensus_meta.cc:287] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.000s
I20260709 13:15:38.669237 11443 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
W20260709 13:15:38.656522 13961 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:15:38.655831 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.680480 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.682315 11445 raft_consensus.cc:2376] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5. Re-sending same reply.
I20260709 13:15:38.675438 13626 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.687947 12065 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:38.681563 11268 catalog_manager.cc:5697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 3, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:15:38.694918 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.698307 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:38.681449 12062 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:38.686584 13864 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.699184 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.706789 11515 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.678452 13949 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.706894 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.707544 11444 raft_consensus.cc:2376] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5. Re-sending same reply.
I20260709 13:15:38.711791 13956 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:38.712232 13956 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:38.712581 13956 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.713581 13862 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:38.714088 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.714865 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.715520 11436 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
W20260709 13:15:38.720144 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 65231.1us
I20260709 13:15:38.721145 11409 leader_election.cc:400] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:38.720491 11268 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.722762 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.723654 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.724241 11519 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:38.728844 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.729496 11445 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:38.731606 11515 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:38.722515 11268 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.703645 13846 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.703357 13839 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:38.742658 13953 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:38.714015 13862 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.744581 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:38.657003 13873 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.735775 11268 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 4 to 5, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 5 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.695101 13818 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.751220 11518 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:38.752737 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.727445 12064 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:38.736974 13949 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.755301 13626 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.759373 13626 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:38.759729 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.760293 11440 raft_consensus.cc:2376] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4. Re-sending same reply.
I20260709 13:15:38.761077 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.761610 11440 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:38.762369 13965 log.cc:927] Time spent T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:15:38.757967 13962 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:38.765178 13649 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:38.765565 13649 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:38.766402 13649 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.766983 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.768939 11438 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:38.770036 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 257679us
I20260709 13:15:38.770376 11368 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:38.770586 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 514) took 1379 ms. Trace:
I20260709 13:15:38.771056 13545 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.770977 11364 rpcz_store.cc:276] 0709 13:15:37.390766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.390879 (+   113us) service_pool.cc:224] Handling call
0709 13:15:38.770376 (+1379497us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b5400249354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:38.770520 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923099392}
I20260709 13:15:38.762428 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:38.774189 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 65987.4us
I20260709 13:15:38.759487 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:38.774283 11440 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:38.778702 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 137803us
I20260709 13:15:38.780045 13711 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:15:38.780412 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.780587 13711 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:38.781592 13711 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:38.755333 13966 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:15:38.783620 11335 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.784060 13649 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.785033 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:38.785862 13964 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:38.786252 13964 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.786578 13964 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.701586 13594 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:38.794000 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 71365.2us
I20260709 13:15:38.791600 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.789798 11268 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:15:38.795018 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.743666 13953 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:38.703734 13952 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.794970 11438 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:15:38.803886 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 138086us
I20260709 13:15:38.793303 11408 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:38.805748 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.806690 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:38.807821 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:38.812829 11516 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:38.813570 13952 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.809219 13969 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.825693 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.826323 11438 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:38.819984 11268 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 2, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.828053 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.827245 11440 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:38.800071 13739 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:15:38.832477 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 73112.6us
W20260709 13:15:38.835793 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 50851us
I20260709 13:15:38.799639 13953 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:38.837234 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.828943 11268 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 3 to 4, term changed from 2 to 3, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68), b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:38.783638 13848 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:15:38.840638 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 66885.1us
W20260709 13:15:38.842779 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 49034.4us
I20260709 13:15:38.824010 13740 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:38.843179 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.843902 11444 raft_consensus.cc:2368] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6.
I20260709 13:15:38.844244 12034 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.846557 11438 raft_consensus.cc:2410] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:38.851227 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:38.852939 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 72067.7us
I20260709 13:15:38.841753 13741 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:15:38.853400 13818 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.844748 13952 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.846670 13873 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.859522 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:38.863020 13741 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:38.856287 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.859136 11440 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:38.865506 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:38.864974 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 58378.3us
I20260709 13:15:38.867727 11519 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:38.858999 13662 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.871632 13912 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.873356 13917 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:15:38.853992 13949 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:38.875962 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:38.854242 13740 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.885347 13956 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.874714 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.890007 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:38.874691 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.881632 11445 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:38.854396 13839 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.884752 13594 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.864212 13741 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:38.840742 13862 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.885423 13760 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:38.885998 13662 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.886229 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.899734 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.869946 13846 raft_consensus.cc:2764] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:15:38.903589 11444 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
W20260709 13:15:38.913283 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68939.6us
I20260709 13:15:38.853585 13969 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:15:38.919310 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.878734 13661 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:38.879710 13917 consensus_queue.cc:1243] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:15:38.913595 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 48966.6us
I20260709 13:15:38.917248 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.916659 11445 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:38.889398 13974 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:15:38.889932 13644 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:38.926934 13818 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:38.929142 13644 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:38.929386 13975 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:38.891865 13949 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:38.931195 13864 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:38.931630 13949 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.919668 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.932381 13864 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:38.932772 13864 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.933314 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.933558 13864 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.934962 13864 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.925457 13661 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:38.938982 13661 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.940228 13790 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.910243 13912 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.942337 13790 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:38.943858 13912 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:38.944069 11365 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:38.926468 13862 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.946344 13975 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.947275 13975 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.947685 11408 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:38.948163 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:38.948467 13818 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:38.949056 13818 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:38.949928 13818 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:38.952623 13975 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.953406 13917 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:38.927347 11438 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:38.887367 13740 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.928529 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.928517 13974 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:38.966130 13974 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:38.967103 13974 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.968140 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.968530 13974 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.969009 11439 raft_consensus.cc:2410] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:38.956167 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.972286 11438 raft_consensus.cc:2410] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:38.972213 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:38.972823 11442 raft_consensus.cc:2376] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6. Re-sending same reply.
I20260709 13:15:38.973325 11517 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:38.973862 11442 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:15:38.974960 11442 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:15:38.976032 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.955372 13661 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:38.920243 11519 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:38.872159 13739 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:15:38.988210 11519 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:38.988999 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:38.998723 11437 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:15:38.923996 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52577.1us
I20260709 13:15:38.888737 13956 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:38.977046 13839 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:39.000838 11409 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:39.002316 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:38.916440 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 74049.3us
W20260709 13:15:38.919759 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 57203.8us
W20260709 13:15:38.978252 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.989390 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:38.991091 13661 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:38.909348 13741 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:38.886349 13953 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.011312 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:38.992100 13963 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:15:38.924645 13760 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:39.013374 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:15:38.993069 11361 consensus_meta.cc:287] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.000s
I20260709 13:15:38.958266 11512 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:38.958773 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:38.956480 11444 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:39.027721 11513 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:38.962486 13764 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:15:38.957167 11445 raft_consensus.cc:2410] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:15:38.962162 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.964699 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:39.035974 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:38.957612 11515 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:38.996021 11440 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:38.992048 11443 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:38.990720 13952 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:38.956176 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:39.014814 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:39.037987 13952 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:39.038308 11510 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:39.039598 11438 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:39.010851 13956 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:15:39.038972 13952 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.042666 13619 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.043685 13839 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.032095 13974 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:39.044724 13974 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:39.014189 13848 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:39.045069 13974 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:39.045358 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.018163 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:39.028512 11436 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:15:39.027709 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:39.025599 13739 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:39.043852 13760 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.035693 13953 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.044966 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:39.045823 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 45099.1us
I20260709 13:15:39.014964 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:39.054605 11437 raft_consensus.cc:2376] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6. Re-sending same reply.
I20260709 13:15:39.050271 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:39.027981 11361 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:39.045812 13348 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.010979 13741 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.043224 13619 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.059485 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:39.044309 13980 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:39.063861 11437 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:39.059167 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:39.063076 11364 pending_rounds.cc:85] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 4
I20260709 13:15:39.064241 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:39.067680 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:39.068907 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.049870 13764 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.067543 13980 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.071931 13980 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:39.072558 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.070729 13626 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:39.074175 13626 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.065477 13739 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:39.074525 13626 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:39.044224 13839 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.075317 13739 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.077039 13839 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:39.057910 13848 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:39.079530 13844 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.079272 13848 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.080005 13844 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.081926 13844 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:39.067823 11364 pending_rounds.cc:107] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:15:39.082443 11364 raft_consensus.cc:2889] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:39.057852 13661 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:39.084048 13661 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:39.084909 13661 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.087101 11438 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:39.087611 13764 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.092020 11437 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:39.092811 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:15:39.087604 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 83596.1us
I20260709 13:15:39.093374 11437 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:15:39.060977 13741 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:39.095554 11517 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:39.096699 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:39.096032 11437 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:39.071712 13594 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:39.100924 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:39.097465 13662 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:39.100831 12068 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:39.100487 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:39.105391 11438 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:39.106869 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:39.107638 11515 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:15:39.037319 13964 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.110634 11514 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:39.067173 13644 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:39.115197 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:39.144881 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 52147.5us
I20260709 13:15:39.097167 13873 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:39.150677 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.148784 11513 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:15:39.153954 13764 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:39.154127 13974 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.117261 13662 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:39.163942 13662 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.164386 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.165323 13348 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:39.166105 13348 consensus_queue.cc:1243] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:39.166589 13348 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:39.167476 13912 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:39.168736 13619 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:39.168906 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:39.168664 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:39.170308 11518 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:39.146402 11370 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:39.193305 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:39.216130 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.228407 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2208) took 1070 ms. Trace:
I20260709 13:15:39.262656 11370 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:39.250331 11518 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:39.273013 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:39.273420 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 200916us
I20260709 13:15:39.273950 11514 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:39.150417 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:39.151439 13844 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:39.167606 13964 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:39.292337 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.236459 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:39.228844 11436 rpcz_store.cc:276] 0709 13:15:38.158403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.978952 (+820549us) service_pool.cc:224] Handling call
0709 13:15:39.028301 (+ 49349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.046535 (+ 18234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.228184 (+181649us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b5400438ad0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:39.228341 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112739712}
I20260709 13:15:39.299072 11335 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:39.300374 12064 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:39.300482 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:39.228314 13912 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:39.228914 13619 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.239630 13953 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:39.165453 13626 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:39.330785 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 211482us
I20260709 13:15:39.228112 13913 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:15:39.147166 13644 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.298134 13649 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:39.335762 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.342650 12034 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:39.328428 13969 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:39.361779 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 168713us
I20260709 13:15:39.339018 13964 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:39.303995 13844 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:39.375142 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 80714.9us
I20260709 13:15:39.333472 13953 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:39.362978 13969 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.392594 13969 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:39.392944 13953 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.394134 13953 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:39.364794 13975 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:39.395592 13975 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:39.396474 13975 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.332441 13619 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.401552 13956 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.404929 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.364931 13790 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:15:39.374742 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 359942us
W20260709 13:15:39.414908 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.445439 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.370152 13964 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:39.406402 13626 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:39.447417 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.447857 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 105268us
W20260709 13:15:39.449376 13979 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 13:15:39.449524 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44672.4us
I20260709 13:15:39.449892 13964 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:39.399916 13974 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:39.460597 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.457813 11335 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.459511 12032 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:39.450275 13644 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:39.462903 11370 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:15:39.463279 11370 pending_rounds.cc:85] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 5
I20260709 13:15:39.463521 11370 pending_rounds.cc:107] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:15:39.463742 11370 raft_consensus.cc:2889] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:39.467301 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:39.468047 11440 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:39.470958 11437 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:39.471359 11437 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:39.473367 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:39.474038 11438 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:39.476855 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:39.444463 13711 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:15:39.501315 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:39.501770 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.502031 11437 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.469331 13848 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:39.480382 13644 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.481151 13740 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.498358 13977 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.446491 13912 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:39.517988 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 73156us
I20260709 13:15:39.522393 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.506708 13839 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:39.531064 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.516914 11512 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
W20260709 13:15:39.545655 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44434.7us
I20260709 13:15:39.524618 13739 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:39.525030 13844 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:39.549300 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 93266.8us
I20260709 13:15:39.553086 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:39.527040 13987 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:39.508562 13990 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:15:39.559127 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:39.562673 13985 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:15:39.586477 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59676.2us
I20260709 13:15:39.588156 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:39.552661 13977 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.563086 13844 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:15:39.561146 13987 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:39.596344 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:39.601750 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 53610us
W20260709 13:15:39.605929 11444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.003s	sys 0.014s
W20260709 13:15:39.595429 13986 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:15:39.592485 13839 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:39.593160 13649 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:39.611897 11440 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:15:39.609139 11444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.003s	sys 0.014s
I20260709 13:15:39.587352 13662 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:15:39.500654 13846 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:15:39.547147 13740 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:15:39.619710 11436 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:15:39.627681 11438 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:39.629745 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:39.630497 11438 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:39.631490 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:39.632027 11438 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:39.607158 13987 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:39.625808 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.630157 11335 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:39.635628 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.636453 11335 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:39.614449 13839 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.640125 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:39.628808 11436 pending_rounds.cc:85] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 3
I20260709 13:15:39.645301 11436 pending_rounds.cc:107] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:15:39.645608 11436 raft_consensus.cc:2889] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:39.624794 13848 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:39.629531 13974 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:39.670492 11440 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:39.606679 13969 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.684348 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:39.685278 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 64443.8us
I20260709 13:15:39.641566 13626 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:39.635358 13619 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:39.670477 13873 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:39.676083 13956 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.692787 11438 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:39.669658 13839 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:39.681883 13974 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.730250 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:39.730273 11438 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:15:39.722846 13980 consensus_meta.cc:287] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.000s
W20260709 13:15:39.738422 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.738858 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.721558 13728 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:39.687636 13594 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:39.743105 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 131855us
W20260709 13:15:39.752362 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 155629us
I20260709 13:15:39.728454 13626 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:39.727110 13873 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:39.753655 12034 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:39.737782 13974 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:39.741279 13728 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:39.759905 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 134161us
I20260709 13:15:39.742297 13739 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:39.766151 11484 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.762557 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:39.757196 13873 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:39.739809 13980 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.754432 13594 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.751523 13647 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:15:39.755987 13848 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:39.762981 13728 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:39.787921 13996 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:15:39.788939 14008 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:39.789496 14008 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:39.782550 11409 leader_election.cc:336] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 13:15:39.791661 14014 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:39.747255 13988 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.178s
I20260709 13:15:39.782660 13626 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.783809 13975 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:39.785149 14012 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:39.792014 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:39.789891 13992 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
W20260709 13:15:39.768240 13983 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:15:39.791601 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:15:39.769963 13995 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:15:39.800943 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1938) took 1259 ms. Trace:
I20260709 13:15:39.801339 11443 rpcz_store.cc:276] 0709 13:15:38.541936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.088308 (+546372us) service_pool.cc:224] Handling call
0709 13:15:39.088644 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.088970 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.088996 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.089405 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.368976 (+279571us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:39.369018 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.369084 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.800756 (+431672us) raft_consensus.cc:1685] finished
0709 13:15:39.800773 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.800889 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6099,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5022,"thread_start_us":279240,"threads_started":1,"wal-append.queue_time_us":415179}
I20260709 13:15:39.802105 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:39.769146 13739 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:39.802955 13998 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:15:39.777082 13989 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:15:39.804463 13975 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:39.807596 14014 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:39.817734 13848 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:39.819509 13994 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:15:39.820358 14011 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:39.821200 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1936) took 1285 ms. Trace:
I20260709 13:15:39.826398 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:39.829615 13980 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:39.830404 13711 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:39.831743 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 78169.1us
I20260709 13:15:39.811537 13953 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.834626 11442 rpcz_store.cc:276] 0709 13:15:38.536020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.071644 (+535624us) service_pool.cc:224] Handling call
0709 13:15:39.071979 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.072332 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.072358 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.072806 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.367609 (+294803us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:39.367649 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.367723 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.789375 (+421652us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:39.820929 (+ 31554us) raft_consensus.cc:1685] finished
0709 13:15:39.820949 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.821144 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1416,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"spinlock_wait_cycles":66872576,"thread_start_us":294478,"threads_started":1,"wal-append.queue_time_us":484990}
I20260709 13:15:39.836520 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.837109 13760 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.837275 13619 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.838462 13739 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:39.836596 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 54107.3us
I20260709 13:15:39.850701 11518 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
W20260709 13:15:39.854605 13999 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.171s	user 0.001s	sys 0.001s
I20260709 13:15:39.835425 14014 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:39.854364 12034 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.862538 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.850088 12064 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:39.865090 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:39.874289 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.875164 11484 leader_election.cc:400] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6.
I20260709 13:15:39.875871 11484 leader_election.cc:403] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:39.852339 13760 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.882925 11442 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:39.852830 13619 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:39.860174 14014 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:39.864750 13969 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:39.869678 14006 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:39.873910 13661 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:39.875588 14012 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:39.879468 13662 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:39.899340 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 93661.6us
W20260709 13:15:39.900815 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63942.2us
I20260709 13:15:39.907269 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:39.910980 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:39.896494 13989 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:39.915966 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:39.907172 14003 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:15:39.916692 12065 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:39.918536 12066 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:39.902613 14012 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.891654 14011 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6.
I20260709 13:15:39.887643 13844 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:15:39.888844 13964 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.906973 13626 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:39.907709 13644 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:39.860370 14008 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.914692 13662 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:39.927307 14012 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:39.962713 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:39.925731 13980 raft_consensus.cc:2764] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:15:39.964243 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56671us
I20260709 13:15:39.899021 13728 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:39.966126 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:39.966799 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:39.931794 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:39.992882 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:39.999862 12062 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:39.992034 14003 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:15:40.000834 12065 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:40.006323 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:40.007568 13988 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:40.008270 14021 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:40.009502 14013 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:40.009917 14013 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:40.010294 14013 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:40.011953 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1930) took 1789 ms. Trace:
I20260709 13:15:39.993599 14011 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:40.012297 11444 rpcz_store.cc:276] 0709 13:15:38.222151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.061449 (+839298us) service_pool.cc:224] Handling call
0709 13:15:39.061784 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.062199 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:39.062523 (+   324us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.616784 (+554261us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.631005 (+ 14221us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.631148 (+   143us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:39.631181 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.631249 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:40.011753 (+380504us) raft_consensus.cc:1685] finished
0709 13:15:40.011774 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.011897 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":180,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":164,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":8980,"thread_start_us":553848,"threads_started":1,"wal-append.queue_time_us":687907}
I20260709 13:15:40.014585 11444 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:39.910872 13975 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:40.012524 14002 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:15:40.045128 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:40.017648 11444 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:39.935689 13626 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:40.009411 14017 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:15:39.999856 12061 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:40.012374 11365 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:40.057276 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:40.009410 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:40.058485 12063 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.068112 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:40.068679 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 62443.1us
I20260709 13:15:40.071895 13728 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:40.067392 13662 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.073042 13846 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:40.075736 13760 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:40.076735 13846 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:40.078359 13846 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:15:40.079784 14019 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:15:40.080493 13846 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:40.081193 13626 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.081660 14011 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:40.082031 14011 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:40.082813 14011 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:40.083765 12066 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:15:40.084262 14022 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:15:40.084268 14011 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.085999 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:40.086159 12066 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:15:40.086809 14020 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:15:40.087497 12066 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:15:40.089203 11443 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:40.090480 11443 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:40.091192 12061 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:40.092694 12062 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:40.078902 14012 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.094930 13846 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:40.096774 14012 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.098956 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.100513 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.101755 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.104007 11440 raft_consensus.cc:2410] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:40.109663 12065 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:40.112344 11444 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:40.114060 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:40.116520 14014 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.117278 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 449) took 1100 ms. Trace:
I20260709 13:15:40.117661 11445 rpcz_store.cc:276] 0709 13:15:39.016740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.112046 (+ 95306us) service_pool.cc:224] Handling call
0709 13:15:39.149671 (+ 37625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.229534 (+ 79863us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:39.302113 (+ 72579us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.609830 (+307717us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.619707 (+  9877us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.619842 (+   135us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:39.620058 (+   216us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.620327 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.620455 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:39.787067 (+166612us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b1800424a20. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:40.117021 (+329954us) raft_consensus.cc:1685] finished
0709 13:15:40.117044 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.117219 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7771,"prepare.queue_time_us":27812,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":84,"raft.queue_time_us":139274,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":27168,"spinlock_wait_cycles":13019520,"thread_start_us":269266,"threads_started":1,"wal-append.queue_time_us":406642}
W20260709 13:15:40.121507 14001 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.558s	user 0.001s	sys 0.002s
I20260709 13:15:40.133885 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:40.200361 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:40.109189 13949 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:40.211906 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 195711us
I20260709 13:15:40.143460 14014 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.247740 11519 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.105008 11362 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:40.224510 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:40.252856 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.067598 14028 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.147s
W20260709 13:15:40.256929 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 234495us
I20260709 13:15:40.257041 11368 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:15:40.248157 13764 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.267765 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:40.269443 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.271234 11365 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:40.272701 13873 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:40.274201 13974 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.000s
I20260709 13:15:40.274590 13974 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:40.275318 13873 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.277231 12068 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:15:40.263682 13739 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.286485 14013 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.266376 13952 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.279592 13764 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.291800 14031 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:40.257508 13949 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.293450 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:40.296157 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:40.296694 11440 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:40.264356 14011 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:15:40.301127 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:40.301338 14011 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:40.301712 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.293011 13764 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.303946 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.304625 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.305258 11366 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:40.305363 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.306164 11367 raft_consensus.cc:2410] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:15:40.306476 13839 consensus_meta.cc:287] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.552s	user 0.007s	sys 0.000s
I20260709 13:15:40.306839 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:40.307304 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.309949 11367 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:40.288182 13964 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:40.300537 11437 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:40.301772 14011 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:40.307621 11436 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:40.307339 14021 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.296307 13952 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.298378 13626 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:40.299680 14023 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.312541 11369 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:40.297246 13739 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:40.313591 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.313676 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.294222 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.321517 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.346179 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.337055 11437 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
W20260709 13:15:40.310650 13987 consensus_meta.cc:287] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.631s	user 0.006s	sys 0.005s
I20260709 13:15:40.337874 11443 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:40.306847 13839 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.338933 11517 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:40.338913 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:40.350402 11514 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:40.293051 14013 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.347289 11437 pending_rounds.cc:85] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 5
I20260709 13:15:40.347556 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.354218 11437 pending_rounds.cc:107] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:15:40.354528 11367 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:40.347692 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.347870 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.354547 11437 raft_consensus.cc:2889] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:40.348397 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:40.357262 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:40.357287 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.361428 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.362699 11436 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:40.364100 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:40.365165 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.362879 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:40.367063 11437 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:40.368309 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:40.348398 14036 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:15:40.369488 13739 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:40.371529 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.372923 11363 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:40.368320 11437 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.374050 13839 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:40.372937 13974 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:40.352531 13987 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:40.375346 11513 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.375866 11437 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:15:40.376391 13987 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:40.378609 13952 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.379277 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:40.338563 14021 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.379850 11363 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:40.382014 14011 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:40.382999 14015 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.552s	user 0.002s	sys 0.000s
I20260709 13:15:40.383217 14011 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:40.385099 14021 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.347193 14023 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.389346 14023 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.390903 13949 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:40.391454 13891 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:15:40.389568 13594 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.392664 13964 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:40.394836 12032 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:40.398159 11370 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:40.398768 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.404853 11363 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:40.407888 11436 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:40.341560 13626 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.381211 14035 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:40.346539 13846 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:15:40.412091 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:40.409106 11370 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:40.417580 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:40.444553 11515 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:40.447455 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 109148us
I20260709 13:15:40.449924 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:40.450516 11370 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:40.453749 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.461354 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:40.464571 11367 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:40.460811 11514 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:40.459807 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:40.476980 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:40.477725 11436 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:40.485514 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.486349 11519 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:40.460276 13980 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 13:15:40.461850 14033 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:15:40.449175 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:40.540113 13984 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.883s	user 0.010s	sys 0.004s
I20260709 13:15:40.542198 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.461474 13873 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:40.464780 14035 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:40.491243 13912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.019s	sys 0.019s
I20260709 13:15:40.488682 14021 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:15:40.594524 14000 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.030s	user 0.000s	sys 0.004s
I20260709 13:15:40.605191 11515 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:15:40.605598 11367 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:40.605902 11519 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: [4.7-5.8]
I20260709 13:15:40.600867 13873 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:40.603811 14021 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:40.609023 13981 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.001s	sys 0.013s
W20260709 13:15:40.609462 13981 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.001s	sys 0.014s
I20260709 13:15:40.609223 14021 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.610332 11362 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
W20260709 13:15:40.610925 14047 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 0.169s	user 0.001s	sys 0.002s
W20260709 13:15:40.611208 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 147517us
I20260709 13:15:40.489310 13974 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
I20260709 13:15:40.585379 14014 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:40.703578 11370 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:15:40.721779 13974 consensus_queue.cc:1243] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:15:40.587716 13912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.020s	sys 0.020s
I20260709 13:15:40.715330 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:40.686244 11518 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.707324 11514 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:40.724148 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:40.685634 11515 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:40.623159 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.723829 13987 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:40.720739 13619 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.698957 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 549) took 1671 ms. Trace:
I20260709 13:15:40.672435 13626 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.612282 13873 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:40.736402 13891 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:40.627954 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:40.668167 13993 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.039s	user 0.009s	sys 0.004s
I20260709 13:15:40.730784 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.740975 13964 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:40.744915 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:40.744555 13891 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.744675 13964 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.745968 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.739622 11364 rpcz_store.cc:276] 0709 13:15:39.027605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.028144 (+   539us) service_pool.cc:224] Handling call
0709 13:15:39.042433 (+ 14289us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:39.090971 (+ 48538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.090999 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:39.109838 (+ 18839us) log.cc:844] Serialized 52 byte log entry
0709 13:15:39.456374 (+346536us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:39.456410 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.964898 (+508488us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b54000a78d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:39.964954 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:40.489375 (+524421us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:40.698703 (+209328us) raft_consensus.cc:1685] finished
0709 13:15:40.698723 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.698871 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7846,"raft.queue_time_us":8106,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":7160,"spinlock_wait_cycles":56138880,"thread_start_us":302257,"threads_started":1,"wal-append.queue_time_us":410151}
I20260709 13:15:40.747594 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.705456 13740 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:40.740922 14011 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:40.706766 13980 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.723644 14014 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.644501 14035 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:40.744356 14043 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:15:40.749464 13740 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:40.754619 13740 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.756373 13980 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.784504 11363 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.789579 13952 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
W20260709 13:15:40.793521 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 398741us
I20260709 13:15:40.796231 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.796813 12066 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:40.751507 13987 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.800336 11514 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:15:40.745604 14008 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:15:40.805500 13740 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:40.808044 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:40.806412 14025 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:40.810143 12063 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:40.810261 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:40.811539 13891 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.786787 13949 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:15:40.831732 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.831986 13964 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.761605 13956 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:15:40.819929 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.832923 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.833998 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:40.834968 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.836011 11364 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.838354 12067 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:40.832881 11519 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:40.809082 13987 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:40.844558 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:40.776362 13989 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:15:40.847317 11519 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:40.849774 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:40.823936 13913 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.822572 14025 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:40.859951 13949 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:40.827322 13644 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.861078 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.861445 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:40.861739 13956 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:40.862314 13989 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:40.862226 14009 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:40.863955 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:40.863955 13952 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:40.865175 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.865525 13913 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.865962 13953 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:40.866307 12060 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.860991 13980 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.867350 11517 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:40.868218 13987 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:40.871441 13728 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.827586 13649 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:40.899866 13649 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.902231 13649 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:40.880574 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:40.903652 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:40.904047 14014 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:40.904687 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.904975 13711 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:40.905365 11516 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:40.890010 13989 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:40.906232 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:40.906351 13989 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:40.888950 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.895035 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:40.907567 13989 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:40.907876 14044 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.299s	user 0.001s	sys 0.002s
I20260709 13:15:40.878821 14025 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:40.878415 13949 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:40.893374 14006 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:40.915077 14006 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:40.915500 11369 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:40.916038 14006 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.897665 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.917778 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.917704 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:40.897156 13891 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:40.889707 11519 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:40.919373 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.899822 13728 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.925268 11443 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:40.929023 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.881893 13956 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:40.914237 12069 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:40.930186 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.919492 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:40.934754 11516 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:40.924453 11515 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:40.938470 11514 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:40.938315 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.904752 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.921447 13711 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:40.930924 13891 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.971500 11519 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:40.893594 13644 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.892584 13619 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:15:41.007915 13619 consensus_queue.cc:1243] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:41.008481 13619 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:41.008814 13619 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:41.009184 13644 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:41.009641 13619 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.010542 13873 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:41.010989 13619 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.012682 13873 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:41.014071 14018 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:41.014535 14018 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:41.015501 14018 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.016558 14018 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.020103 13728 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:41.020341 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.973955 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.021143 12061 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:41.021632 12067 raft_consensus.cc:2410] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:15:41.022393 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.023063 12067 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:15:40.898151 13953 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.975946 13956 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:40.980203 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:40.981451 13997 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.419s	user 0.001s	sys 0.003s
I20260709 13:15:40.981626 12069 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:15:40.982280 14004 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.206s	user 0.002s	sys 0.001s
I20260709 13:15:40.983645 11512 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:40.922405 13917 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:15:40.938625 13987 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:15:40.964327 13952 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.054688 12069 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:40.974581 13711 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:40.939144 14013 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
W20260709 13:15:41.032652 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 173540us
I20260709 13:15:41.040902 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:41.043944 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 148972us
W20260709 13:15:41.044154 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 64503us
I20260709 13:15:41.048235 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 700) took 1138 ms. Trace:
I20260709 13:15:41.081945 12068 rpcz_store.cc:276] 0709 13:15:39.901539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.901668 (+   129us) service_pool.cc:224] Handling call
0709 13:15:39.902022 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:40.277627 (+375605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.039572 (+761945us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b540052b7d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:41.039748 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862324480}
W20260709 13:15:41.061829 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.082890 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:41.076262 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:41.024523 13891 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.014647 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:40.983670 13913 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:41.085213 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:41.086894 11514 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:41.090860 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:41.095043 11511 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:41.097029 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:41.097640 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:41.099191 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.101845 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.102689 12065 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:41.096060 11519 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
W20260709 13:15:41.105996 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 65154us
I20260709 13:15:41.108019 13649 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:41.108534 13649 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:41.109256 11510 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:15:41.109704 11510 pending_rounds.cc:85] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 3
I20260709 13:15:41.109335 13649 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.109972 11510 pending_rounds.cc:107] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:15:41.110224 11510 raft_consensus.cc:2889] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:41.111059 14011 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:41.111878 14065 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:41.113059 13949 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.113651 14065 raft_consensus.cc:2764] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:15:41.115571 14057 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:41.116346 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:41.116935 11515 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:41.117964 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:41.118562 11368 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:41.121944 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:41.122572 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:41.123687 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:41.125483 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:41.125711 11442 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:41.126065 11367 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:15:41.127856 13913 consensus_queue.cc:1059] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:41.128057 14009 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:41.128841 13913 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:41.129269 14009 raft_consensus.cc:3055] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:41.129622 14009 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:41.130427 14009 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.133276 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:41.136299 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:41.105528 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:41.145323 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:41.145932 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48338.8us
I20260709 13:15:41.122546 11361 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
W20260709 13:15:41.162117 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 56666.7us
I20260709 13:15:41.159024 11514 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:41.160837 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.161104 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1949) took 1706 ms. Trace:
I20260709 13:15:41.165129 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:41.145416 13956 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.167235 11438 rpcz_store.cc:276] 0709 13:15:39.454403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.737906 (+283503us) service_pool.cc:224] Handling call
0709 13:15:39.738214 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.756274 (+ 18060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.756307 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.763445 (+  7138us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.778944 (+ 15499us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:39.779893 (+   949us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.780068 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.382368 (+602300us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:41.160793 (+778425us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b540028a5d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:41.160853 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.160899 (+    46us) raft_consensus.cc:1685] finished
0709 13:15:41.160913 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.161048 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8507,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":31,"raft.queue_time_us":77,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"spinlock_wait_cycles":46200320,"thread_start_us":1135,"threads_started":2,"wal-append.queue_time_us":26521}
I20260709 13:15:41.171934 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:41.172120 11367 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:15:41.172988 11368 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:15:41.179142 11368 pending_rounds.cc:85] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 5
I20260709 13:15:41.179567 11368 pending_rounds.cc:107] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:15:41.149524 13848 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:41.156224 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 105362us
I20260709 13:15:41.143958 13662 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.177335 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.178965 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:41.149470 13969 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.153189 14007 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:41.178960 11367 pending_rounds.cc:85] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 6
I20260709 13:15:41.182049 11368 raft_consensus.cc:2889] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:41.192569 11367 pending_rounds.cc:107] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:15:41.156080 13647 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:41.191896 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.192955 11367 raft_consensus.cc:2889] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:41.151144 14011 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:41.194800 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:41.175967 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 151290us
I20260709 13:15:41.158315 14009 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:41.197773 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.202602 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:41.211032 11438 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.212854 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:41.202543 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:41.205104 11442 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:41.205142 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:41.214743 11445 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:41.169310 13873 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.208648 13728 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:41.213440 11366 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:41.176203 14065 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:41.208387 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 125571us
I20260709 13:15:41.233707 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:41.221552 11438 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:41.229843 13891 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:41.192480 13848 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.198201 14008 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.217815 13739 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:41.164999 13644 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:15:41.265058 11409 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:41.149706 13975 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:15:41.163853 14027 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:15:41.259500 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:41.270493 11442 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:41.235230 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:41.218325 13987 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:41.192233 13846 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:15:41.274840 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:41.275532 11370 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:41.237219 14025 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.277644 14025 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:41.257485 13848 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:41.238883 13952 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:41.212517 13662 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.261432 13739 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.286808 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:41.268787 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 118742us
I20260709 13:15:41.218106 13647 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.258971 14073 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:41.301429 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:41.302109 11438 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:41.305418 11438 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:41.307896 11438 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:41.266610 12060 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:41.294782 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:41.299546 13739 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:41.324075 14067 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:15:41.332206 13739 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.333990 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:41.299714 13964 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:41.241427 14065 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:41.312537 12061 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:41.300822 13987 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:41.314623 14071 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.314818 13953 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:41.361270 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:41.366521 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 171160us
I20260709 13:15:41.380386 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:41.354796 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 144538us
I20260709 13:15:41.339883 13969 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:41.523099 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:41.615747 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 350736us
I20260709 13:15:41.616972 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:41.525131 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.618327 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:41.601552 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 399034us
I20260709 13:15:41.619751 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:41.299294 13952 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.620993 11333 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:41.617581 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:41.615141 11369 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:41.341415 14073 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:41.623307 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.633363 11409 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:41.635996 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:41.637401 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:41.354571 13964 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.351269 13739 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:41.366351 13649 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:41.684593 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 464970us
I20260709 13:15:41.373013 13912 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.268s
I20260709 13:15:41.642618 11366 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:41.353505 13662 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:41.366181 13977 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.350383 13913 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.290s
I20260709 13:15:41.511279 14008 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:41.340039 14080 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:41.635444 14071 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.665369 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:41.665961 14027 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:41.667078 14073 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.668110 13987 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.672302 13891 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:41.673759 13953 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.640125 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:41.681288 13644 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:41.642669 13964 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.355970 14011 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:41.622803 13969 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.631639 14057 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:41.689302 11517 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:41.721958 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1834) took 1406 ms. Trace:
W20260709 13:15:41.719111 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.722916 11513 rpcz_store.cc:276] 0709 13:15:40.312479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.312604 (+   125us) service_pool.cc:224] Handling call
0709 13:15:40.321148 (+  8544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:40.375691 (+ 54543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.718784 (+1343093us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b540031e554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:41.718941 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124279168}
I20260709 13:15:41.690392 14080 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.693993 14008 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.689266 13649 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.721704 13969 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:41.777526 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.725523 13739 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:41.782987 13912 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:41.808553 13912 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.809682 14086 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:15:41.811036 13912 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:41.792627 14009 consensus_meta.cc:287] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.000s
I20260709 13:15:41.795713 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:41.814965 14018 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.798s	user 0.007s	sys 0.003s
I20260709 13:15:41.815178 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.815339 13949 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:41.796054 13873 consensus_meta.cc:287] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.003s
W20260709 13:15:41.802426 14053 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.877s	user 0.003s	sys 0.010s
W20260709 13:15:41.774822 14030 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:41.823258 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.829380 12062 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:41.830297 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.830855 12062 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:41.831180 11442 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.000s
I20260709 13:15:41.807837 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 707) took 1888 ms. Trace:
I20260709 13:15:41.788812 14071 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:41.725960 14080 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.811273 14086 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.742038 13891 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.811946 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.843017 12066 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:41.811789 14057 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:41.812451 13956 consensus_meta.cc:287] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.001s
I20260709 13:15:41.750833 11361 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:41.787386 14075 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:41.818084 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:41.818645 13873 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.767212 14072 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:41.822417 13848 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.000s
I20260709 13:15:41.816069 13647 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:41.825327 13846 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:41.826591 13662 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:41.872911 11370 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:15:41.760756 14066 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.831570 11442 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:15:41.829496 11518 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:15:41.695806 13977 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.834199 14071 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:41.835048 13989 consensus_meta.cc:287] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.927s	user 0.009s	sys 0.001s
I20260709 13:15:41.761981 14036 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:15:41.837023 14082 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:41.832726 12064 rpcz_store.cc:276] 0709 13:15:39.919515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.928357 (+  8842us) service_pool.cc:224] Handling call
0709 13:15:40.617593 (+689236us) spinlock_profiling.cc:243] Waited 6.95 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:40.617815 (+   222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:40.823734 (+205919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:40.827513 (+  3779us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:40.848901 (+ 21388us) log.cc:844] Serialized 26 byte log entry
0709 13:15:40.980214 (+131313us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:40.980266 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.980345 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.806184 (+825839us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b1800267f40. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:41.807590 (+  1406us) raft_consensus.cc:1685] finished
0709 13:15:41.807617 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.807771 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":188422,"raft.queue_time_us":203577,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6957,"spinlock_wait_cycles":29603456,"thread_start_us":115464,"threads_started":1,"wal-append.queue_time_us":124883}
I20260709 13:15:41.838589 12068 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:41.839958 14091 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:15:41.840626 14089 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:41.827303 14007 consensus_meta.cc:287] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.613s	user 0.007s	sys 0.001s
I20260709 13:15:41.841995 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2182) took 3726 ms. Trace:
I20260709 13:15:41.873135 13662 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:41.721642 13953 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:41.864007 11515 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:15:41.864989 12067 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
W20260709 13:15:41.869166 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 360996us
I20260709 13:15:41.921504 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2227) took 1811 ms. Trace:
I20260709 13:15:41.888949 14084 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:41.865671 14014 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:41.866466 13956 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.866214 13891 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:41.872149 13848 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:41.874135 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 241165us
I20260709 13:15:41.875660 11518 pending_rounds.cc:85] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 6
W20260709 13:15:41.874061 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 236725us
I20260709 13:15:41.852182 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 531) took 1054 ms. Trace:
I20260709 13:15:41.869941 14075 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:41.874935 14008 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:41.909243 11439 rpcz_store.cc:276] 0709 13:15:38.115324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.970436 (+855112us) service_pool.cc:224] Handling call
0709 13:15:38.970733 (+   297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:38.993954 (+ 23221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.993987 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:39.097730 (+103743us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:39.167138 (+ 69408us) log.cc:844] Serialized 52 byte log entry
0709 13:15:39.632596 (+465458us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:39.801829 (+169233us) spinlock_profiling.cc:243] Waited 6.25 ms on lock 0x7b18000308e0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35bc5a64bc 00007f35b98f448b 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf
0709 13:15:39.801944 (+   115us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.802129 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.802193 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:40.704451 (+902258us) spinlock_profiling.cc:243] Waited 891 us on lock 0x7b18000308e0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:40.931826 (+227375us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:41.838621 (+906795us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b240040c8c8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b97979c4 00007f35b9795acb 00007f35b9950dc9 00007f35b9950a55 00007f35b994ffb0 00007f35bc5c6488 00007f35bc5c6be8 00007f35bc59e398 00007f35bc59976a 00007f35bc598bb6
0709 13:15:41.841778 (+  3157us) raft_consensus.cc:1685] finished
0709 13:15:41.841798 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.841932 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":154589,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":23741,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":194,"spinlock_wait_cycles":166696960,"thread_start_us":530631,"threads_started":2,"wal-append.queue_time_us":573882}
I20260709 13:15:41.923610 11436 rpcz_store.cc:276] 0709 13:15:40.110140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.541696 (+431556us) service_pool.cc:224] Handling call
0709 13:15:40.587243 (+ 45547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:40.591789 (+  4546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:40.591817 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:40.619755 (+ 27938us) log.cc:844] Serialized 548 byte log entry
0709 13:15:40.759161 (+139406us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:40.759192 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.759255 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.801598 (+1042343us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:41.912519 (+110921us) raft_consensus.cc:1685] finished
0709 13:15:41.912540 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.921444 (+  8904us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4875008,"thread_start_us":113934,"threads_started":1,"wal-append.queue_time_us":229104}
I20260709 13:15:41.850373 13739 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.921767 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:41.854244 14086 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:42.257431 11511 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.006s	sys 0.022s
W20260709 13:15:42.330552 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 408820us
I20260709 13:15:42.332768 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:42.125959 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.198455 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.215754 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:42.335170 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.306018 11511 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.008s	sys 0.022s
I20260709 13:15:41.975253 11444 raft_consensus.cc:2410] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:15:42.333619 12066 rpcz_store.cc:276] 0709 13:15:40.797347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.806734 (+  9387us) service_pool.cc:224] Handling call
0709 13:15:41.109200 (+302466us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:41.811800 (+702600us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9e9c 00007f35b98d9a5b 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:41.852106 (+ 40306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69986816}
W20260709 13:15:42.215538 11483 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.004s	sys 0.033s
W20260709 13:15:42.350523 11483 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.005s	sys 0.033s
W20260709 13:15:42.187273 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.905617 14071 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:42.360817 11366 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:42.374861 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.360040 14089 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:42.305915 14084 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:41.970896 13964 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.372852 13647 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:15:41.966997 13873 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:42.376317 14100 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:42.375161 11518 pending_rounds.cc:107] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 5.7
W20260709 13:15:41.970688 13619 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.935s	user 0.004s	sys 0.004s
I20260709 13:15:41.912762 14007 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:42.295699 14092 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 13:15:42.329427 13739 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:41.969105 13649 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:41.966235 14087 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:42.387478 11518 raft_consensus.cc:2889] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:42.379455 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:41.966871 12064 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:41.968461 13977 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.206903 13913 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:42.490669 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:42.422816 11334 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.008s	sys 0.032s
I20260709 13:15:42.372717 13953 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.372551 14099 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:42.391286 11438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.005s	sys 0.027s
I20260709 13:15:42.436626 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.438007 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:42.376137 14066 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.179594 13949 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:42.385584 14100 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.380391 14072 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:42.383791 14086 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.381621 14075 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:42.494405 11334 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.008s	sys 0.032s
I20260709 13:15:42.376323 13974 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:42.494338 11368 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:42.304425 14080 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.376333 14014 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.436843 14011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.007s	sys 0.009s
I20260709 13:15:42.385630 13891 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.449282 14073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.011s	sys 0.023s
W20260709 13:15:42.488938 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 109575us
I20260709 13:15:42.381263 13662 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.497593 11438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.006s	sys 0.028s
W20260709 13:15:42.437193 14023 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.507s	user 0.017s	sys 0.015s
W20260709 13:15:42.495862 14057 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.011s	sys 0.017s
I20260709 13:15:42.384372 13873 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.430862 14069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.008s	sys 0.028s
I20260709 13:15:42.389556 13917 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:42.379732 13956 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
W20260709 13:15:42.494247 11512 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.007s	sys 0.020s
I20260709 13:15:42.385051 14093 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.131s
W20260709 13:15:42.502957 13988 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.011s	sys 0.018s
W20260709 13:15:42.506590 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 819468us
I20260709 13:15:42.508649 12033 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:42.496497 11510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.000s	sys 0.011s
I20260709 13:15:42.501943 14009 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.492605 14102 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:42.514446 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.383663 13964 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.507103 13949 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:42.517414 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 562) took 1346 ms. Trace:
W20260709 13:15:42.523092 14069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.008s	sys 0.029s
I20260709 13:15:42.493283 13649 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
W20260709 13:15:42.584213 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 69840.5us
I20260709 13:15:42.526928 13917 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.524480 13980 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:42.507596 13974 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:42.503607 14100 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:42.382411 14084 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:42.513679 14011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.008s	sys 0.011s
I20260709 13:15:42.509392 14086 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.433159 14103 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:15:42.497927 13913 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.516271 14073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.011s	sys 0.023s
I20260709 13:15:42.380944 14089 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:42.518230 14023 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.591s	user 0.020s	sys 0.015s
W20260709 13:15:42.526224 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 151428us
I20260709 13:15:42.593330 13980 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:42.520073 14057 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.012s	sys 0.017s
W20260709 13:15:42.530238 11510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.000s	sys 0.011s
I20260709 13:15:42.529225 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 569) took 1166 ms. Trace:
I20260709 13:15:42.535804 11363 rpcz_store.cc:276] 0709 13:15:41.170968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.171149 (+   181us) service_pool.cc:224] Handling call
0709 13:15:41.200283 (+ 29134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.200619 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:41.200647 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.216589 (+ 15942us) log.cc:844] Serialized 26 byte log entry
0709 13:15:41.720462 (+503873us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:41.720499 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.720569 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.517209 (+796640us) raft_consensus.cc:1685] finished
0709 13:15:42.517230 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.517357 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1176,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":7544,"thread_start_us":440494,"threads_started":1,"wal-append.queue_time_us":603738}
I20260709 13:15:42.489529 13846 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 13:15:42.528065 11512 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.007s	sys 0.020s
W20260709 13:15:42.530709 13988 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.011s	sys 0.018s
I20260709 13:15:42.503393 14066 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.528885 13848 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.529810 13662 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.501976 14099 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.602949 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.588326 14007 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:42.507534 14072 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:41.895484 12060 raft_consensus.cc:2955] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:42.592353 13975 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:42.594628 13987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.045s	sys 0.054s
W20260709 13:15:42.527154 14082 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.007s	sys 0.011s
W20260709 13:15:42.652124 14082 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.008s	sys 0.012s
I20260709 13:15:42.651754 13975 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.510119 14071 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.654850 14107 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:42.593034 14075 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:42.478495 14087 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:42.616699 11511 rpcz_store.cc:276] 0709 13:15:41.362497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.362640 (+   143us) service_pool.cc:224] Handling call
0709 13:15:41.616023 (+253383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.616235 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:42.337284 (+721049us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.527901 (+190617us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.527957 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:42.528002 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.528073 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.529152 (+  1079us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":272941,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":911051,"threads_started":2}
I20260709 13:15:42.627851 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:42.588654 14104 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:42.643386 13846 consensus_queue.cc:1243] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:42.638883 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.683166 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:42.683382 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44606us
I20260709 13:15:42.656050 13975 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.684636 13953 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:42.686666 13917 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:42.688552 14014 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.684793 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:42.691809 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:42.692952 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:42.633682 14106 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:42.652143 14108 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:15:42.644364 14057 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:42.617651 13980 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.612408 14011 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:15:42.651676 13987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.045s	sys 0.054s
I20260709 13:15:42.652462 14082 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:42.657096 11368 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:42.585373 14009 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.679611 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 63339.3us
I20260709 13:15:42.690371 13891 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.722276 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2239) took 1514 ms. Trace:
I20260709 13:15:42.612473 14103 consensus_queue.cc:1243] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:42.644285 14073 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:42.644850 14007 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:42.684252 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.693492 11511 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:15:42.702733 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.693298 14099 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:42.706557 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.611246 14086 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.722738 11438 rpcz_store.cc:276] 0709 13:15:41.207488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.310776 (+103288us) service_pool.cc:224] Handling call
0709 13:15:41.311041 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.335970 (+ 24929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.336004 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.336489 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:15:42.516946 (+1180457us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:42.516991 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.517052 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.722025 (+204973us) raft_consensus.cc:1685] finished
0709 13:15:42.722048 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.722203 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24551,"raft.queue_time_us":62044,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":90673,"thread_start_us":1126094,"threads_started":1,"wal-append.queue_time_us":1171206}
I20260709 13:15:42.682284 14080 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:42.705469 14109 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:15:42.740208 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.749637 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:42.750174 11367 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:42.743728 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 59562.9us
I20260709 13:15:42.751961 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.752755 14071 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:42.753180 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.753875 14071 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:42.754282 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.754949 14071 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:42.744562 14117 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:15:42.746111 13891 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:42.743146 14007 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.755334 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:42.780005 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.747116 14086 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.781713 14071 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:42.783308 11511 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:42.788247 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.721727 14023 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.784443 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.805279 11363 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:42.807080 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:42.807792 11511 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.740556 14014 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:42.717149 14011 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.705653 14057 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.763928 13913 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:42.780787 14093 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:42.840762 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.842084 11363 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:42.803582 14117 consensus_queue.cc:1243] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:15:42.843652 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 51890.5us
I20260709 13:15:42.846030 14093 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:42.787822 14071 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:42.848491 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:42.807560 13956 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:42.853644 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.847530 14089 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.812971 13949 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.804541 14073 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:42.811967 14102 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:42.871238 11367 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:42.871789 14125 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.868072 14071 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.882632 14125 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.873477 11511 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:42.840907 14086 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:42.881369 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.762836 14100 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.893832 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:42.885397 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:42.894390 11363 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:42.891151 14086 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:42.898703 14086 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.900486 14125 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.901546 11369 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:42.891682 13969 consensus_meta.cc:287] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 1.075s	user 0.001s	sys 0.010s
I20260709 13:15:42.890576 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.721853 14106 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.811190 14023 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.879828 13952 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.840173 14014 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.905881 11364 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:42.881678 14089 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:42.906553 11363 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:15:42.910319 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.852365 13891 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:15:42.910749 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:42.911230 11366 raft_consensus.cc:2955] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:42.917472 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:42.918541 14118 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:15:42.918812 14062 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:42.932088 14062 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:42.915948 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 574) took 1072 ms. Trace:
I20260709 13:15:42.919869 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:42.922518 14100 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.923142 14012 raft_consensus.cc:2955] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 3 to 6, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:42.840794 14082 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:42.926087 11444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 464) took 1802 ms (client timeout 1855 ms). Trace:
I20260709 13:15:42.933199 14023 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:42.941581 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:42.913944 14093 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.933751 11510 rpcz_store.cc:276] 0709 13:15:41.843525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.843643 (+   118us) service_pool.cc:224] Handling call
0709 13:15:41.866028 (+ 22385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:41.872423 (+  6395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.872455 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:41.887538 (+ 15083us) log.cc:844] Serialized 52 byte log entry
0709 13:15:42.616733 (+729195us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:42.616778 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.616840 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.914454 (+297614us) raft_consensus.cc:1685] finished
0709 13:15:42.914472 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.915885 (+  1413us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109354,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":696255,"threads_started":1,"wal-append.queue_time_us":829505}
I20260709 13:15:42.910032 14087 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:42.911734 13969 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.953758 14057 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:42.955961 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:42.928645 14116 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:15:42.957962 11444 rpcz_store.cc:269] 0709 13:15:41.123174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.123482 (+   308us) service_pool.cc:224] Handling call
0709 13:15:42.925844 (+1802362us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400386654. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:42.926025 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956616448}
I20260709 13:15:42.961706 14106 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:42.957947 13739 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.004s
I20260709 13:15:42.933066 14062 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.979626 11367 pending_rounds.cc:85] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 3
I20260709 13:15:43.028158 11367 pending_rounds.cc:107] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:15:43.028561 11367 raft_consensus.cc:2889] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:43.005558 12065 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:43.010686 14087 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.030807 14087 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:42.977571 12063 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:43.044291 14071 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.048087 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:43.049414 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 570) took 1686 ms. Trace:
I20260709 13:15:43.022174 14011 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.053534 13969 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:43.064342 11517 rpcz_store.cc:276] 0709 13:15:41.363132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.725949 (+362817us) service_pool.cc:224] Handling call
0709 13:15:41.734994 (+  9045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.803693 (+ 68699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.803723 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:41.803765 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:41.803796 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.049176 (+1245380us) spinlock_profiling.cc:243] Waited 68.5 ms on lock 0x7b540031f454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:43.049223 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.049359 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157511936}
I20260709 13:15:43.070431 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:43.071019 13760 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:43.071348 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:43.072057 13760 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:43.072412 13760 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:43.020205 14072 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:42.978048 13739 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.065228 14062 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:43.090044 11365 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:43.093324 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.074276 14084 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:43.093930 14084 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:43.094746 14084 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.095744 14106 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:43.096319 14106 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:43.086898 13760 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:43.097218 13974 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:43.093488 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 567) took 1916 ms. Trace:
I20260709 13:15:43.105531 11365 rpcz_store.cc:276] 0709 13:15:41.176768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.180180 (+  3412us) service_pool.cc:224] Handling call
0709 13:15:43.047919 (+1867739us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:43.093428 (+ 45509us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41592576}
I20260709 13:15:43.103560 13980 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.106879 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:43.106179 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 543) took 2128 ms. Trace:
I20260709 13:15:43.111922 11364 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:43.110831 12061 rpcz_store.cc:276] 0709 13:15:40.978057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.049731 (+ 71674us) service_pool.cc:224] Handling call
0709 13:15:41.155040 (+105309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.373061 (+218021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.105893 (+1732832us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540070f3d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:43.106114 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289521408}
I20260709 13:15:43.105804 13974 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:43.091935 12063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 719) took 1963 ms (client timeout 1569 ms). Trace:
I20260709 13:15:43.119935 14065 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:43.102986 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 990) took 1247 ms. Trace:
I20260709 13:15:43.097132 14106 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.107913 13980 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.123075 13760 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:43.123777 14129 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:15:43.124788 14087 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:43.123340 14127 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.144s	user 0.003s	sys 0.002s
I20260709 13:15:43.101454 14007 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:43.144275 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.158691 14021 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:43.126080 12063 rpcz_store.cc:269] 0709 13:15:41.123508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.126883 (+  3375us) service_pool.cc:224] Handling call
0709 13:15:42.919717 (+1792834us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:15:43.076269 (+156552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26859904}
I20260709 13:15:43.143635 13953 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:43.164719 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2122) took 1542 ms. Trace:
I20260709 13:15:43.151891 14072 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.165364 11370 rpcz_store.cc:276] 0709 13:15:41.622564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.622816 (+   252us) service_pool.cc:224] Handling call
0709 13:15:41.623140 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.873329 (+250189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.164456 (+1291127us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400249354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:43.164660 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573914368}
I20260709 13:15:43.209789 11365 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:43.212217 13644 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:43.229274 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.229938 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.230772 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.231544 11484 connection.cc:256] RPC call timeout handler was delayed by -1.1583140349998757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:15:43.231997 11484 connection.cc:256] RPC call timeout handler was delayed by -1.0989140349997797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
I20260709 13:15:43.168143 13953 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.232788 13980 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.169981 14072 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:43.233318 13980 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:43.235144 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1970) took 2840 ms. Trace:
I20260709 13:15:43.235535 14072 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:43.235872 13980 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:43.204994 14071 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.235575 11437 rpcz_store.cc:276] 0709 13:15:40.394757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.925778 (+531021us) service_pool.cc:224] Handling call
0709 13:15:40.930742 (+  4964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.183664 (+252922us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:41.216296 (+ 32632us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.056613 (+1840317us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b1800492fa0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e23a 00007f35bca08a2a
0709 13:15:43.137487 (+ 80874us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.138958 (+  1471us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.139122 (+   164us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:43.139157 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.139236 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:43.234834 (+ 95598us) raft_consensus.cc:1685] finished
0709 13:15:43.234932 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.235081 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38300,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":5035,"raft.queue_time_us":231106,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":28866,"spinlock_wait_cycles":3847424,"thread_start_us":325887,"threads_started":2,"wal-append.queue_time_us":1987791}
I20260709 13:15:43.240056 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:43.241082 11519 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:43.236502 14013 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.244170 14013 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.129942 12060 rpcz_store.cc:276] 0709 13:15:41.855603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.894734 (+ 39131us) service_pool.cc:224] Handling call
0709 13:15:41.895054 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.895281 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:42.953216 (+1057935us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.065402 (+112186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.068910 (+  3508us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:43.074979 (+  6069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.087252 (+ 12273us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.098177 (+ 10925us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":271,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":8836,"thread_start_us":83540,"threads_started":1,"wal-append.queue_time_us":77552}
W20260709 13:15:43.586236 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.615960 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.023s	sys 0.018s
W20260709 13:15:43.251097 14133 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 13:15:43.319634 12065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 720) took 2195 ms (client timeout 1808 ms). Trace:
W20260709 13:15:43.766172 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.025s	sys 0.025s
I20260709 13:15:43.246114 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:43.385048 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:43.261125 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:43.719720 11408 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.630s (SENT) [suppressed 19 similar messages]
W20260709 13:15:43.659210 14138 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:15:43.658929 14128 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.618s	user 0.002s	sys 0.000s
I20260709 13:15:43.661554 11370 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:43.360839 11437 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:15:43.804491 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.805639 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.806576 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.291044 14018 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:43.251578 12034 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.101s	user 0.032s	sys 0.051s
W20260709 13:15:43.809255 12034 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.658s	user 0.045s	sys 0.070s
W20260709 13:15:43.809937 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 2.65897e+06us
I20260709 13:15:43.717945 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:43.386693 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.811685 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.812274 11438 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:43.813138 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:43.269948 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.777076 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.385434 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.814424 11439 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:43.260759 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 567) took 1901 ms. Trace:
I20260709 13:15:43.815223 11518 rpcz_store.cc:276] 0709 13:15:41.359595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.359765 (+   170us) service_pool.cc:224] Handling call
0709 13:15:41.360083 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.482592 (+1122509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.260502 (+777910us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54000e8b54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:43.260695 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078711680}
W20260709 13:15:43.796008 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 947522us
W20260709 13:15:43.752420 14062 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.000s
W20260709 13:15:43.543516 11363 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.008s	sys 0.019s
W20260709 13:15:43.817410 11363 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.018s	sys 0.022s
W20260709 13:15:43.288741 14079 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.954s	user 0.007s	sys 0.010s
I20260709 13:15:43.659348 14120 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.337248 13953 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:43.782224 12065 rpcz_store.cc:269] 0709 13:15:41.123715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.522778 (+399063us) service_pool.cc:224] Handling call
0709 13:15:43.319364 (+1796586us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400599054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:43.319540 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3192721280}
I20260709 13:15:43.247529 14066 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.259827 13739 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:43.290228 12068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 722) took 2166 ms (client timeout 1588 ms). Trace:
W20260709 13:15:43.833225 12068 rpcz_store.cc:269] 0709 13:15:41.124139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.157465 (+ 33326us) service_pool.cc:224] Handling call
0709 13:15:43.290016 (+2132551us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54005f73d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:43.290133 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511826176}
W20260709 13:15:43.772101 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:43.791103 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 69521.2us
I20260709 13:15:43.835186 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:43.835711 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.836388 11439 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:43.287972 14071 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:43.823529 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:43.360486 14072 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.374873 14089 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.255777 14014 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.769070 14057 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.436484 13661 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:43.827303 11437 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:43.828264 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.830948 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:43.831666 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:43.660815 14083 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:43.813913 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.800420 14082 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:15:43.778295 14075 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:43.838333 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.839684 14120 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.840274 13974 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:43.840495 11365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:43.841336 14066 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.842303 13848 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:43.897800 13848 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:43.797585 13952 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.859833 13912 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.869805 11518 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:43.260373 14007 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:43.898219 13848 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.827515 14013 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.887527 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.889605 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.907773 11445 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:43.908824 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.866906 14098 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:43.886438 14106 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.018s	sys 0.030s
I20260709 13:15:43.893407 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.913300 11439 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:15:43.797298 14080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.018s	sys 0.061s
I20260709 13:15:43.869019 14014 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:43.827620 14018 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.903627 11365 raft_consensus.cc:606] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:43.867079 14043 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:43.870368 13839 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:43.871496 11519 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:43.909444 11445 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:43.920573 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.907462 11438 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:43.921782 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.922585 11438 raft_consensus.cc:2410] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:15:43.910504 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:43.862179 13953 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:15:43.862733 14089 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:43.860986 14071 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.872481 14057 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.917027 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:43.890049 13661 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.827566 14025 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 13:15:43.912986 14106 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.018s	sys 0.030s
I20260709 13:15:43.923906 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:43.915740 14080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.018s	sys 0.061s
I20260709 13:15:43.935319 14080 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:43.933856 11514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.010s	sys 0.014s
W20260709 13:15:43.935801 13891 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.002s	sys 0.053s
I20260709 13:15:43.860754 14072 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:43.934190 14088 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.007s	sys 0.021s
I20260709 13:15:43.894383 14075 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:43.919796 14014 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.907418 13912 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.931566 14043 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.900496 14066 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:43.941082 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:43.985839 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 585) took 1228 ms. Trace:
I20260709 13:15:43.990639 11519 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:43.942946 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:43.902148 13952 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.926530 11365 raft_consensus.cc:614] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:43.933579 11515 raft_consensus.cc:2410] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:43.984854 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2229) took 2782 ms. Trace:
I20260709 13:15:43.944095 11436 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:43.934068 14065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.011s	sys 0.013s
W20260709 13:15:43.918341 14012 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.023s	sys 0.032s
I20260709 13:15:43.946871 11445 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:43.912988 14098 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.984563 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2127) took 1259 ms. Trace:
I20260709 13:15:43.967118 14089 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:43.962524 13848 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:43.954270 14075 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:43.966024 14057 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.977873 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 573) took 2134 ms. Trace:
I20260709 13:15:43.966775 14071 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:43.978227 11470 heartbeater.cc:646] Failed to heartbeat to 127.10.4.126:42409 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.10.4.126:42409 timed out after 15.000s (SENT)
I20260709 13:15:43.979794 14043 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.983721 14083 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:43.991348 11369 rpcz_store.cc:276] 0709 13:15:42.757058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.902280 (+145222us) service_pool.cc:224] Handling call
0709 13:15:42.902554 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:42.946447 (+ 43893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.946483 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:42.947071 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.107846 (+160775us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b18002a0e20. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:15:43.529077 (+421231us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:43.529118 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.985558 (+456440us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b5400259254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:43.985602 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:43.985643 (+    41us) raft_consensus.cc:1685] finished
0709 13:15:43.985655 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.985788 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5704,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4374,"spinlock_wait_cycles":99941120,"thread_start_us":420944,"threads_started":1,"wal-append.queue_time_us":1033000}
I20260709 13:15:44.001977 14093 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.954113 14120 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:43.936288 11514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.010s	sys 0.014s
I20260709 13:15:44.003726 11439 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:15:44.001531 14066 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:44.003938 11518 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:43.944458 13891 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.002s	sys 0.053s
I20260709 13:15:44.157310 11519 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
W20260709 13:15:44.157413 14012 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.034s	sys 0.036s
I20260709 13:15:43.953837 14072 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:43.949497 14088 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.007s	sys 0.022s
W20260709 13:15:44.004307 13790 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.003s	sys 0.027s
I20260709 13:15:44.152192 11443 rpcz_store.cc:276] 0709 13:15:41.202082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.203169 (+  1087us) service_pool.cc:224] Handling call
0709 13:15:41.203490 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.085114 (+1881624us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b1800200440. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35bc5a64bc 00007f35b98f448b 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35bc9f6060 00007f35bc9f6bfb 00007f35bca0164b
0709 13:15:43.085269 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.085295 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.085713 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.528409 (+442696us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:43.528452 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.528967 (+   515us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:43.984611 (+455644us) raft_consensus.cc:1685] finished
0709 13:15:43.984627 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.984803 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108730,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":36722,"spinlock_wait_cycles":2577152,"thread_start_us":600997,"threads_started":2,"wal-append.queue_time_us":895957}
I20260709 13:15:44.156898 11366 rpcz_store.cc:276] 0709 13:15:42.724963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.898943 (+173980us) service_pool.cc:224] Handling call
0709 13:15:42.899203 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.911104 (+ 11901us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:42.953593 (+ 42489us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.027769 (+ 74176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.027833 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:43.027880 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.984338 (+956458us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b54002db754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:43.984383 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.984507 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":957,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":2658,"spinlock_wait_cycles":26910464,"thread_start_us":72846,"threads_started":1,"wal-append.queue_time_us":74671,"wal-append.run_cpu_time_us":1059,"wal-append.run_wall_time_us":1037028}
I20260709 13:15:43.961562 14014 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:43.969167 13912 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:44.150868 14065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.019s	sys 0.016s
I20260709 13:15:44.193265 13952 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:44.197854 11437 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:15:44.151010 14083 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:44.179889 11518 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:44.199342 11437 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:15:44.218284 11437 pending_rounds.cc:85] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 3
I20260709 13:15:44.218581 11437 pending_rounds.cc:107] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:15:44.218838 11437 raft_consensus.cc:2889] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:44.071564 11334 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.534s (SENT)
I20260709 13:15:44.190948 14120 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.227983 14120 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:44.177549 14093 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:44.232043 14083 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:44.242964 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:44.190868 13790 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.003s	sys 0.037s
W20260709 13:15:44.333032 11510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.027s	sys 0.038s
W20260709 13:15:44.333588 11510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.027s	sys 0.039s
W20260709 13:15:44.314153 11362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.164s	user 0.009s	sys 0.055s
W20260709 13:15:44.344542 11362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.194s	user 0.009s	sys 0.055s
W20260709 13:15:44.346994 11444 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.009s	user 0.014s	sys 0.040s
W20260709 13:15:44.347497 11444 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.014s	sys 0.040s
I20260709 13:15:44.358803 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1861) took 1757 ms. Trace:
I20260709 13:15:44.359333 11514 rpcz_store.cc:276] 0709 13:15:42.601124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.055089 (+453965us) service_pool.cc:224] Handling call
0709 13:15:43.055570 (+   481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:43.060494 (+  4924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.060522 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:43.068212 (+  7690us) log.cc:844] Serialized 574 byte log entry
0709 13:15:44.158370 (+1090158us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:44.158407 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.158475 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.358526 (+200051us) raft_consensus.cc:1685] finished
0709 13:15:44.358548 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.358728 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6659,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":4869,"thread_start_us":1055045,"threads_started":1,"wal-append.queue_time_us":1268978}
I20260709 13:15:44.377568 14009 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:44.467840 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:44.473666 11484 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.013s	sys 0.046s
W20260709 13:15:44.474238 11484 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.013s	sys 0.046s
W20260709 13:15:44.474663 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 722538us
I20260709 13:15:44.474123 11514 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:15:44.475099 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.016s	sys 0.029s
W20260709 13:15:44.475484 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.016s	sys 0.029s
W20260709 13:15:44.476259 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.005s	sys 0.037s
W20260709 13:15:44.476626 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.005s	sys 0.037s
I20260709 13:15:44.477977 11518 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:15:44.479272 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 660095us
I20260709 13:15:44.480065 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:44.502965 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:44.504733 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:44.505431 11518 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:44.470057 14006 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.516779 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:44.517679 11440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.016s	sys 0.031s
W20260709 13:15:44.518041 11440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.016s	sys 0.031s
I20260709 13:15:44.508522 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:44.501643 14043 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:44.509240 11518 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:44.514107 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.530601 11369 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:44.515892 13989 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:44.546556 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.530263 14084 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.547964 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2209) took 6223 ms. Trace:
I20260709 13:15:44.526981 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:44.583379 11364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 587) took 1645 ms (client timeout 1988 ms). Trace:
W20260709 13:15:44.583899 11364 rpcz_store.cc:269] 0709 13:15:42.938013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.941343 (+  3330us) service_pool.cc:224] Handling call
0709 13:15:44.583138 (+1641795us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400423e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:44.583309 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346559872}
I20260709 13:15:44.518319 14006 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:44.554060 11441 rpcz_store.cc:276] 0709 13:15:38.158737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.990163 (+831426us) service_pool.cc:224] Handling call
0709 13:15:38.994245 (+  4082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.994438 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.994459 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.102809 (+108350us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:39.103153 (+   344us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.461798 (+358645us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:39.461830 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.461889 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:40.848963 (+1387074us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:43.208804 (+2359841us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b240040c8c8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b97979c4 00007f35b9795acb 00007f35b9950dc9 00007f35b9950a55 00007f35b994ffb0 00007f35bc5c6488 00007f35bc5c6be8 00007f35bc59e398 00007f35bc59976a 00007f35bc598bb6
0709 13:15:43.209431 (+   627us) raft_consensus.cc:1685] finished
0709 13:15:43.209450 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.382122 (+1172672us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b54000834d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:44.382268 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74119936,"thread_start_us":358312,"threads_started":1,"wal-append.queue_time_us":458426,"wal-append.run_cpu_time_us":24334,"wal-append.run_wall_time_us":4655710}
I20260709 13:15:44.530761 14043 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:44.639894 12034 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.002s	sys 0.009s
W20260709 13:15:44.640367 12034 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.002s	sys 0.009s
W20260709 13:15:44.642539 11363 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.000s	sys 0.011s
W20260709 13:15:44.642961 11363 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.000s	sys 0.011s
W20260709 13:15:44.647863 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 100644us
I20260709 13:15:44.654709 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 589) took 1708 ms. Trace:
I20260709 13:15:44.657347 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:44.656737 11363 rpcz_store.cc:276] 0709 13:15:42.946057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.953960 (+  7903us) service_pool.cc:224] Handling call
0709 13:15:42.954479 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.954663 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:43.817852 (+863189us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.643255 (+825403us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.654373 (+ 11118us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:44.654420 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.654509 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.654650 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32946,"prepare.queue_time_us":59724,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":38,"raft.queue_time_us":80239,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6473,"thread_start_us":1654833,"threads_started":2}
I20260709 13:15:44.657977 11441 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:44.658237 11518 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:44.665498 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 185500us
W20260709 13:15:44.666309 11408 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.000s	sys 0.060s
W20260709 13:15:44.666704 11408 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.000s	sys 0.061s
W20260709 13:15:44.667006 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 947269us
W20260709 13:15:44.541869 11442 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.201s	user 0.027s	sys 0.040s
W20260709 13:15:44.741909 11442 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.401s	user 0.035s	sys 0.041s
W20260709 13:15:44.735899 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 912488us
W20260709 13:15:44.741309 11436 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.003s
I20260709 13:15:44.736445 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:44.736502 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 664955us
I20260709 13:15:44.752754 11436 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:15:44.762818 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:44.818284 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 74062.5us
W20260709 13:15:44.828357 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 97381.9us
I20260709 13:15:44.176527 11512 rpcz_store.cc:276] 0709 13:15:41.843088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.845750 (+  2662us) service_pool.cc:224] Handling call
0709 13:15:41.901556 (+ 55806us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:42.642066 (+740510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.645671 (+  3605us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:42.689807 (+ 44136us) log.cc:844] Serialized 52 byte log entry
0709 13:15:42.844587 (+154780us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:42.897023 (+ 52436us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.914356 (+ 17333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.920538 (+  6182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:43.513942 (+593404us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:43.927570 (+413628us) raft_consensus.cc:1685] finished
0709 13:15:43.941851 (+ 14281us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.977790 (+ 35939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10504,"prepare.queue_time_us":26734,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":77616,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":30032,"spinlock_wait_cycles":15631744,"thread_start_us":837188,"threads_started":2,"wal-append.queue_time_us":203437}
I20260709 13:15:44.829270 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.751510 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.829082 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:44.834672 11484 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:44.835645 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.830343 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.658715 13711 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:44.857813 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:44.859175 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.859822 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.851649 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:44.861096 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.889959 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:44.588207 14084 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:44.864655 11512 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:15:44.897017 11515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 577) took 2916 ms (client timeout 2599 ms). Trace:
I20260709 13:15:44.837713 14009 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.920970 14009 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:44.852268 13790 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:44.897727 11515 rpcz_store.cc:269] 0709 13:15:41.980769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.073155 (+ 92386us) service_pool.cc:224] Handling call
0709 13:15:44.882153 (+2808998us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540014c3d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:44.896945 (+ 14792us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3672618752}
I20260709 13:15:44.921653 11369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:44.861285 14071 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:44.832137 14057 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:15:44.928987 11369 raft_consensus.cc:606] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:44.928987 11436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1989) took 1584 ms (client timeout 1566 ms). Trace:
I20260709 13:15:44.928916 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 600) took 1074 ms. Trace:
W20260709 13:15:44.923607 13594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.014s	sys 0.041s
I20260709 13:15:44.860366 13873 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:44.928496 11438 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:44.897181 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.935603 11368 rpcz_store.cc:276] 0709 13:15:43.854564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.854780 (+   216us) service_pool.cc:224] Handling call
0709 13:15:44.928271 (+1073491us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:44.928448 (+   177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.928611 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.928638 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.928677 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:44.928706 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.928757 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.928864 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12194048}
I20260709 13:15:44.861083 14012 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:15:44.906388 14165 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.927786 14065 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.945178 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.949688 11368 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:44.935299 11369 raft_consensus.cc:614] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:15:44.951848 11511 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.021s	sys 0.031s
I20260709 13:15:44.923830 13844 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:44.935705 11436 rpcz_store.cc:269] 0709 13:15:43.344932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.384812 (+ 39880us) service_pool.cc:224] Handling call
0709 13:15:44.928143 (+1543331us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002797d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:44.928917 (+   774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341452544}
I20260709 13:15:44.934897 14161 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.954473 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:44.958209 11436 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:44.958706 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:44.959479 11368 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:44.959359 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:44.955596 11511 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.021s	sys 0.031s
W20260709 13:15:44.938692 13594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.014s	sys 0.042s
W20260709 13:15:44.948521 13891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.019s	sys 0.040s
I20260709 13:15:44.941915 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:44.961012 11437 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.025s	sys 0.036s
W20260709 13:15:44.962733 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.101s	user 0.011s	sys 0.024s
I20260709 13:15:44.968073 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:44.962160 14021 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.048s	user 0.013s	sys 0.021s
W20260709 13:15:44.969779 11482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.099s	user 0.030s	sys 0.078s
I20260709 13:15:44.970750 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:44.975139 11444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.015s	sys 0.039s
W20260709 13:15:44.979864 11437 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.026s	sys 0.036s
I20260709 13:15:44.956871 14120 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:44.953361 13913 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.992s	user 0.008s	sys 0.026s
W20260709 13:15:44.980007 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.129s	user 0.012s	sys 0.024s
I20260709 13:15:44.962353 14097 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:44.974543 13891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.020s	sys 0.040s
W20260709 13:15:44.986884 11482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.031s	sys 0.078s
W20260709 13:15:44.986781 11334 leader_election.cc:336] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.569s (SENT)
I20260709 13:15:44.975883 13917 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.935377 13790 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:44.963584 13848 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:44.956977 13661 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:44.939270 14071 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:44.986817 11444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.015s	sys 0.039s
I20260709 13:15:44.974082 14009 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:44.994927 11334 leader_election.cc:336] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.588s (SENT)
I20260709 13:15:44.980909 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:44.962239 14161 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:44.996017 11513 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.087s	user 0.026s	sys 0.019s
I20260709 13:15:44.979984 14098 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:44.985607 14021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.013s	sys 0.021s
I20260709 13:15:44.952602 14165 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:44.954685 14065 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:44.960397 13844 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:44.991866 13974 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.091s	user 0.015s	sys 0.027s
W20260709 13:15:45.026970 11483 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.017s	sys 0.036s
W20260709 13:15:45.059288 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 64397.4us
W20260709 13:15:45.042104 11513 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.027s	sys 0.019s
I20260709 13:15:45.037277 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1988) took 1692 ms. Trace:
I20260709 13:15:45.076972 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 1995) took 1248 ms. Trace:
I20260709 13:15:45.067251 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:45.078094 11368 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:45.077816 11445 rpcz_store.cc:276] 0709 13:15:43.828350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.920342 (+ 91992us) service_pool.cc:224] Handling call
0709 13:15:45.076814 (+1156472us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b5400208fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:45.076919 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59079424}
W20260709 13:15:45.017509 14155 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.436s	user 0.000s	sys 0.001s
I20260709 13:15:45.037876 11443 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:15:45.025275 13848 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:45.077447 11483 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.212s	user 0.017s	sys 0.045s
W20260709 13:15:44.991581 13913 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.008s	sys 0.027s
W20260709 13:15:44.997437 14088 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.035s	sys 0.035s
I20260709 13:15:45.100629 11366 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:15:45.071439 11517 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.293s	user 0.025s	sys 0.028s
I20260709 13:15:45.062127 14018 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.077668 11439 rpcz_store.cc:276] 0709 13:15:43.344706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.964538 (+619832us) service_pool.cc:224] Handling call
0709 13:15:43.964781 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.158791 (+194010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.037028 (+878237us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b540027a1d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:45.037222 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88812288}
I20260709 13:15:45.106884 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2240) took 1312 ms. Trace:
I20260709 13:15:45.116061 11438 rpcz_store.cc:276] 0709 13:15:43.794299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.194113 (+399814us) service_pool.cc:224] Handling call
0709 13:15:44.928264 (+734151us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:44.938346 (+ 10082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.106673 (+168327us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54002797d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:45.106819 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687663360}
I20260709 13:15:45.117406 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:45.017356 14149 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.428s	user 0.001s	sys 0.000s
I20260709 13:15:44.997036 13594 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:45.067188 14156 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
W20260709 13:15:45.059942 14062 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.096s	user 0.030s	sys 0.051s
I20260709 13:15:45.076870 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:45.016040 14162 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:15:45.101239 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 1.23559e+06us
I20260709 13:15:45.192728 11436 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:15:45.079504 11334 leader_election.cc:336] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.665s (SENT)
I20260709 13:15:45.024896 14164 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:45.102084 11517 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.335s	user 0.025s	sys 0.028s
I20260709 13:15:45.027671 14043 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.079916 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:45.195674 11334 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:45.078680 13974 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.177s	user 0.022s	sys 0.033s
I20260709 13:15:45.199520 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1856) took 2763 ms. Trace:
W20260709 13:15:45.199756 14163 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:15:45.199888 14106 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:44.998013 14097 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.201062 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1978) took 1863 ms. Trace:
I20260709 13:15:45.200125 11510 rpcz_store.cc:276] 0709 13:15:42.436455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.964374 (+527919us) service_pool.cc:224] Handling call
0709 13:15:42.979146 (+ 14772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:42.979496 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:42.979522 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.013368 (+ 33846us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.318003 (+304635us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b18002dee00. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:15:44.334025 (+1016022us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:44.334068 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.334151 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.199169 (+865018us) raft_consensus.cc:1685] finished
0709 13:15:45.199188 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.199449 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":460,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":6429,"spinlock_wait_cycles":31485696,"thread_start_us":1015798,"threads_started":1,"wal-append.queue_time_us":1540110}
I20260709 13:15:45.202294 14097 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:45.202842 14176 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:15:45.061662 13891 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:45.025375 14093 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:45.101900 14088 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.038s	sys 0.036s
I20260709 13:15:45.104970 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:15:45.105455 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:45.206928 11367 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:45.017531 14006 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:45.066375 14009 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.061582 14073 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:45.122398 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:45.102594 14152 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:15:45.212636 11364 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:15:45.084259 14147 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.175999 14172 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:45.102191 14084 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.179867 11438 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:15:45.180418 11439 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:15:45.001919 13917 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:45.196743 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 117270us
I20260709 13:15:45.232210 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:45.233789 11334 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.756s (SENT)
W20260709 13:15:45.197285 11483 connection.cc:256] RPC call timeout handler was delayed by -1.0680622260001655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:15:45.198453 14178 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:15:45.198036 14164 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.201480 11440 rpcz_store.cc:276] 0709 13:15:43.337911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.341068 (+  3157us) service_pool.cc:224] Handling call
0709 13:15:43.386055 (+ 44987us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:43.738749 (+352694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.738781 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:43.779074 (+ 40293us) log.cc:844] Serialized 574 byte log entry
0709 13:15:44.551950 (+772876us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:44.551991 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.552065 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.200869 (+648804us) raft_consensus.cc:1685] finished
0709 13:15:45.200887 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.201007 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57112,"raft.queue_time_us":352156,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":14170,"thread_start_us":713621,"threads_started":1,"wal-append.queue_time_us":1148562}
W20260709 13:15:45.190888 14062 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.227s	user 0.030s	sys 0.051s
I20260709 13:15:45.178879 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:45.219695 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2241) took 1425 ms. Trace:
I20260709 13:15:45.223255 11439 pending_rounds.cc:85] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 7
I20260709 13:15:45.224032 11363 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:45.214919 14043 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.246166 11439 pending_rounds.cc:107] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:15:45.246953 11439 raft_consensus.cc:2889] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:45.245929 11443 rpcz_store.cc:276] 0709 13:15:43.794566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.194373 (+399807us) service_pool.cc:224] Handling call
0709 13:15:45.037612 (+843239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:45.097563 (+ 59951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.219498 (+121935us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b540027a1d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:45.219632 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1938674560}
I20260709 13:15:45.207635 13891 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.215196 14021 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.245954 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:45.191165 14162 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:45.103869 13848 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:45.222956 14172 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.190469 13594 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.258054 11439 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:15:45.258904 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:45.213754 14009 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:45.261586 11442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.075s	sys 0.098s
I20260709 13:15:45.244114 14027 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:45.217056 14073 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.223119 14147 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:45.267232 11442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.076s	sys 0.098s
I20260709 13:15:45.267992 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 566) took 4091 ms. Trace:
I20260709 13:15:45.261507 14172 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:45.268723 14065 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.260321 14083 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:45.252981 13891 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:45.274151 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1977) took 1936 ms. Trace:
I20260709 13:15:45.256131 13844 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:45.265100 14071 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.272706 13594 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.251817 14043 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.271957 14027 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:45.256444 14021 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:45.284262 14180 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:15:45.307610 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:45.308462 12065 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:45.311652 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:45.311550 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.312290 12069 raft_consensus.cc:2376] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7. Re-sending same reply.
I20260709 13:15:45.314741 14013 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:45.315217 14013 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:45.315629 14013 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:45.322471 11268 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39444 (request call id 351) took 16332 ms (client timeout 15000 ms). Trace:
I20260709 13:15:45.324637 14073 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.311996 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:45.325743 12060 raft_consensus.cc:2376] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5. Re-sending same reply.
I20260709 13:15:45.326421 14009 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:45.327796 13790 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:45.309845 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:45.291812 14161 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:45.335770 12065 raft_consensus.cc:2376] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6. Re-sending same reply.
I20260709 13:15:45.324553 14082 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:45.382710 12062 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:45.385255 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.386188 12062 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:45.332458 14164 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:45.333941 14027 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.393326 14027 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:45.390684 11516 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:15:45.398125 11516 pending_rounds.cc:85] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 3
I20260709 13:15:45.398388 11516 pending_rounds.cc:107] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:15:45.372985 14147 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.398618 11516 raft_consensus.cc:2889] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:45.388648 14073 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:45.398557 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:45.400313 12060 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:45.276021 11362 rpcz_store.cc:276] 0709 13:15:41.176460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.197107 (+ 20647us) service_pool.cc:224] Handling call
0709 13:15:41.260938 (+ 63831us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.683956 (+423018us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:41.686268 (+  2312us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.149648 (+1463380us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b2c000c7fc8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:43.150100 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:15:44.346620 (+1196520us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:44.346661 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.901008 (+554347us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b540035c354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:44.913317 (+ 12309us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.267735 (+354418us) raft_consensus.cc:1685] finished
0709 13:15:45.267758 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.267906 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3080,"raft.queue_time_us":103383,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":15871,"spinlock_wait_cycles":183159552,"thread_start_us":1196096,"threads_started":1,"wal-append.queue_time_us":1397162}
W20260709 13:15:45.402207 13989 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.856s	user 0.009s	sys 0.001s
I20260709 13:15:45.402681 13989 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:45.399230 11516 pending_rounds.cc:107] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:15:45.403820 11516 raft_consensus.cc:2889] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:45.331358 11262 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 3 to 6, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 6 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.403468 11445 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:45.381954 14161 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.383885 14082 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:45.303897 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 452283us
I20260709 13:15:45.413591 11262 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 5, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 5 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.421732 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:45.304427 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 436637us
I20260709 13:15:45.305887 14185 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:45.267938 14165 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:45.291472 14162 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.264099 13848 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:45.396840 14182 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:45.426664 11262 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 3 to 4, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:45.476578 11262 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 5, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 5 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.405301 14008 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.492015 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.492933 12062 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:45.496302 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 1972) took 2159 ms. Trace:
I20260709 13:15:45.496872 11444 rpcz_store.cc:276] 0709 13:15:43.336372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.337682 (+  1310us) service_pool.cc:224] Handling call
0709 13:15:43.338002 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.338201 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:44.347924 (+1009723us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.495831 (+1147907us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.495899 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:45.495960 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.496047 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.496217 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1038354,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":201180,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":93,"thread_start_us":2156916,"threads_started":3,"wal-append.queue_time_us":751305}
W20260709 13:15:45.420171 11518 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.032s	sys 0.036s
W20260709 13:15:45.514112 11518 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.033s	sys 0.036s
W20260709 13:15:45.529007 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50316.1us
I20260709 13:15:45.478271 11262 catalog_manager.cc:5697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:15:45.535027 11262 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 5, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 5 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.536865 11262 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 3 to 6, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:15:45.544385 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:45.577072 11262 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 3 to 4, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 4 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.545121 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.546682 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.547034 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.550614 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.552094 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.555552 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.556370 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:45.556720 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:45.559954 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:45.559996 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 576) took 3690 ms. Trace:
I20260709 13:15:45.420693 14161 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:45.428156 13989 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:15:45.601194 11482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.019s	sys 0.031s
W20260709 13:15:45.601608 11482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.019s	sys 0.031s
W20260709 13:15:45.601939 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:45.573369 13873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.011s	sys 0.003s
W20260709 13:15:45.566175 11437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.007s	sys 0.008s
I20260709 13:15:45.488909 14147 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:45.607442 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1852) took 4515 ms. Trace:
W20260709 13:15:45.322926 11268 rpcz_store.cc:269] 0709 13:15:28.998210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.131299 (+133089us) service_pool.cc:224] Handling call
0709 13:15:30.415198 (+1283899us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54001897d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc968090 00007f35bca06f3a 00007f35c27bd1cc 00007f35c28bb37e 00007f35c0d7c2a4 00007f35c0d7c232 00007f35c0d7c161 00007f35c0d7c0dc 00007f35c0d7b392 00007f35bad3948c 00007f35bad388c6
0709 13:15:45.322359 (+14907161us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:39.533287 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:39.670176 (+136889us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:39.680664 (+ 10488us) write_op.cc:435] Acquired schema lock
0709 13:15:39.683251 (+  2587us) tablet.cc:662] Decoding operations
0709 13:15:39.828050 (+144799us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:39.828126 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:39.828154 (+    28us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:15:39.829438 (+  1284us) tablet.cc:701] Row locks acquired
0709 13:15:39.829487 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:15:39.829668 (+   181us) write_op.cc:270] Start()
0709 13:15:39.829840 (+   172us) write_op.cc:276] Timestamp: P: 1783602939829595 usec, L: 0
0709 13:15:39.829870 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:15:39.830591 (+   721us) log.cc:844] Serialized 5601 byte log entry
0709 13:15:40.420295 (+589704us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35b98f40ac 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b
0709 13:15:40.919932 (+499637us) op_driver.cc:464] REPLICATION: finished
0709 13:15:41.156596 (+236664us) write_op.cc:301] APPLY: starting
0709 13:15:41.175823 (+ 19227us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:41.278257 (+102434us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:41.278300 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:41.818160 (+539860us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:41.818538 (+   378us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:15:41.818592 (+    54us) write_op.cc:312] APPLY: finished
0709 13:15:41.819061 (+   469us) log.cc:844] Serialized 101 byte log entry
0709 13:15:42.600635 (+781574us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:42.613034 (+ 12399us) write_op.cc:454] Released schema lock
0709 13:15:42.625042 (+ 12008us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:42.687419 (+ 62377us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:42.687454 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":51,"reported_tablets":26,"spinlock_wait_cycles":1889560320,"child_traces":[["op",{"apply.queue_time_us":211599,"apply.run_cpu_time_us":42747,"apply.run_wall_time_us":1517593,"num_ops":14,"prepare.queue_time_us":578199,"prepare.run_cpu_time_us":19802,"prepare.run_wall_time_us":1098172,"raft.queue_time_us":216962,"raft.run_cpu_time_us":14052,"raft.run_wall_time_us":180357,"replication_time_us":1089893,"spinlock_wait_cycles":5550464,"thread_start_us":2184584,"threads_started":5,"wal-append.queue_time_us":1294308,"wal-append.run_cpu_time_us":30213,"wal-append.run_wall_time_us":2566113}]]}
I20260709 13:15:45.403535 14073 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:45.572594 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.007s	sys 0.008s
W20260709 13:15:45.617123 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.007s	sys 0.008s
I20260709 13:15:45.543926 13917 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:45.597223 14012 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.009s	sys 0.007s
I20260709 13:15:45.634013 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.626466 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:45.638023 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:45.652281 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:45.609795 14014 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.006s	sys 0.011s
I20260709 13:15:45.444662 14185 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:45.627353 14057 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.001s	sys 0.026s
I20260709 13:15:45.281345 11442 rpcz_store.cc:276] 0709 13:15:43.337650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.340150 (+  2500us) service_pool.cc:224] Handling call
0709 13:15:43.340461 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.340679 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:44.743443 (+1402764us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.273808 (+530365us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.273863 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:45.273921 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.273989 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.274102 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1036047,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":356,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":2443,"thread_start_us":1932385,"threads_started":2,"wal-append.queue_time_us":536461}
I20260709 13:15:45.563076 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.565204 14162 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:45.544900 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 285254us
I20260709 13:15:45.595696 11513 rpcz_store.cc:276] 0709 13:15:41.869250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.869396 (+   146us) service_pool.cc:224] Handling call
0709 13:15:43.908705 (+2039309us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800010b98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:43.908919 (+   214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:45.077966 (+1169047us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:45.077996 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:45.079222 (+  1226us) log.cc:844] Serialized 26 byte log entry
0709 13:15:45.488088 (+408866us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:45.488127 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.488200 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.488242 (+    42us) raft_consensus.cc:1685] finished
0709 13:15:45.488252 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.559934 (+ 71682us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":247533,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":21520,"spinlock_wait_cycles":24283392,"thread_start_us":1477963,"threads_started":2,"wal-append.queue_time_us":361995}
I20260709 13:15:45.595878 11436 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:45.540210 14182 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:45.597111 12068 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:45.595852 12067 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:15:45.604827 11437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.008s	sys 0.008s
W20260709 13:15:45.605041 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.025s	sys 0.025s
I20260709 13:15:45.561473 14008 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:45.679131 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2246) took 1876 ms. Trace:
I20260709 13:15:45.610736 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:45.611008 12033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.011s	sys 0.009s
W20260709 13:15:45.682227 12033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.011s	sys 0.010s
W20260709 13:15:45.682565 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 600703us
W20260709 13:15:45.611868 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 187746us
I20260709 13:15:45.597762 11262 catalog_manager.cc:5697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:15:45.650987 14021 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:45.572362 11333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.009s	sys 0.005s
W20260709 13:15:45.689849 11333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.010s	sys 0.005s
W20260709 13:15:45.661895 14057 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.001s	sys 0.027s
W20260709 13:15:45.617614 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 719908us
W20260709 13:15:45.602319 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.625926 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:45.659816 13790 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:45.692219 11443 raft_consensus.cc:2410] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:15:45.602677 13873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.011s	sys 0.003s
I20260709 13:15:45.660499 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:45.696818 12066 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:45.697875 12066 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:45.580313 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1865) took 2898 ms. Trace:
I20260709 13:15:45.659068 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:45.562387 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 327764us
I20260709 13:15:45.655037 12064 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:45.674266 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.629431 13989 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:45.629210 14012 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.009s	sys 0.007s
W20260709 13:15:45.678334 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.025s	sys 0.025s
W20260709 13:15:45.718724 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59731.8us
W20260709 13:15:45.656486 14014 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.006s	sys 0.011s
W20260709 13:15:45.609901 11511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.003s	sys 0.016s
W20260709 13:15:45.740387 11511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.003s	sys 0.016s
I20260709 13:15:45.607848 11516 rpcz_store.cc:276] 0709 13:15:41.086002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.086111 (+   109us) service_pool.cc:224] Handling call
0709 13:15:41.086376 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:45.404368 (+4317992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.601062 (+196694us) spinlock_profiling.cc:243] Waited 4.3 s on lock 0x7b54003d8e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:45.601214 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9898176896}
I20260709 13:15:45.654510 13917 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:45.685384 11445 raft_consensus.cc:2410] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:45.569489 14082 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:45.627920 14147 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:45.695281 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:45.745880 11514 raft_consensus.cc:2410] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:15:45.699625 11334 leader_election.cc:336] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.808s (SENT)
I20260709 13:15:45.715155 12060 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:45.699028 11517 rpcz_store.cc:276] 0709 13:15:42.681601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.071326 (+389725us) service_pool.cc:224] Handling call
0709 13:15:43.071620 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.072294 (+   674us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:43.072631 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.718465 (+645834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.765903 (+ 47438us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.766409 (+   506us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:45.195951 (+1429542us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.579979 (+384028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.580055 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.580093 (+    38us) raft_consensus.cc:1685] finished
0709 13:15:45.580103 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.580252 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49733,"prepare.queue_time_us":1317622,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":3014,"raft.queue_time_us":288716,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":40067,"thread_start_us":2168926,"threads_started":3,"wal-append.queue_time_us":869307,"wal-append.run_cpu_time_us":1845,"wal-append.run_wall_time_us":1019464}
I20260709 13:15:45.702605 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:45.748108 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 579) took 1919 ms. Trace:
I20260709 13:15:45.661650 14206 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:45.684476 11262 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 5, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 5 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.709117 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:45.753283 12068 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:45.713567 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:45.714533 11515 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:45.682624 11437 rpcz_store.cc:276] 0709 13:15:43.802893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.220371 (+417478us) service_pool.cc:224] Handling call
0709 13:15:44.220713 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.220952 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:44.986998 (+766046us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.678783 (+691785us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.678835 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:45.678877 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.678981 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.679083 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":155952,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":3042,"thread_start_us":1457112,"threads_started":2}
I20260709 13:15:45.717062 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:45.686270 14043 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:45.720980 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:45.739646 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:45.671519 14006 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:45.746577 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46983.5us
I20260709 13:15:45.748677 11511 rpcz_store.cc:276] 0709 13:15:43.828671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.828814 (+   143us) service_pool.cc:224] Handling call
0709 13:15:43.863499 (+ 34685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.863808 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:44.960700 (+1096892us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.747089 (+786389us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.747258 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.747303 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:45.747345 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.747951 (+   606us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.748058 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4728,"prepare.queue_time_us":1263009,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"thread_start_us":1852149,"threads_started":2}
I20260709 13:15:45.754217 11262 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 5, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:45.708022 14208 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.845206 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:45.851965 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:45.757195 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 85090.9us
I20260709 13:15:45.757691 12066 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:45.744920 13913 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:45.744165 13917 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.752462 14167 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:45.748345 13953 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:45.757669 14082 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:45.758349 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 55808.2us
I20260709 13:15:45.931890 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:45.872031 11519 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:45.800258 11262 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.877456 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1874) took 1057 ms. Trace:
W20260709 13:15:45.882057 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 162655us
I20260709 13:15:45.744192 14021 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.123s
W20260709 13:15:45.866564 11334 leader_election.cc:336] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.824s (SENT)
I20260709 13:15:45.759222 14009 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 13:15:45.867843 14083 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.746106 14210 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.136s
I20260709 13:15:45.757944 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:45.848505 13790 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:15:45.979671 14027 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.586s	user 0.005s	sys 0.009s
I20260709 13:15:45.981621 14083 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:45.934326 11262 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.850431 14165 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 13:15:45.939821 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 256207us
I20260709 13:15:45.852097 14012 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:15:45.950182 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:45.988466 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 121946us
I20260709 13:15:45.932812 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:45.902262 14198 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:15:45.889293 14200 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:45.933693 11518 rpcz_store.cc:276] 0709 13:15:44.819776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.819959 (+   183us) service_pool.cc:224] Handling call
0709 13:15:44.820249 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.820507 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:45.514555 (+694048us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.872759 (+358204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:45.872810 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:45.872851 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.872909 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.877384 (+  4475us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20777,"prepare.queue_time_us":402068,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2956,"raft.queue_time_us":167097,"spinlock_wait_cycles":2499840,"thread_start_us":999898,"threads_started":3,"wal-append.queue_time_us":335506}
I20260709 13:15:46.004766 12062 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:45.919538 14184 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:45.937666 13917 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:45.927948 14065 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:45.924326 14073 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:45.968669 14013 consensus_meta.cc:287] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.653s	user 0.003s	sys 0.007s
W20260709 13:15:46.019001 11334 leader_election.cc:336] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.854s (SENT)
W20260709 13:15:46.019469 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 85425.6us
I20260709 13:15:46.020138 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:46.020884 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.021489 11518 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:15:46.022339 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.022734 11518 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
W20260709 13:15:46.024880 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 305539us
I20260709 13:15:46.029259 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:45.987085 11262 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 3 to 2, leader changed from <none> to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67), b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from VOTER to NON_VOTER, now has a pending config: VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65), VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66), VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67), VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 3 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:46.004880 14165 consensus_queue.cc:1243] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:46.038156 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.044798 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.046636 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.047266 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.014770 14161 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.048374 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:45.952096 13953 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.047173 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.052475 11518 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:46.054836 11511 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:45.918202 14208 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.050927 11262 catalog_manager.cc:5697] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 1 to 3, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 3 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:46.053009 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:46.055757 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.081602 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.082836 11511 raft_consensus.cc:2410] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:15:45.968046 13989 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.096669 11445 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:46.097944 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.056478 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.099015 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.099752 11436 raft_consensus.cc:2410] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:15:46.100100 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:46.100559 11444 raft_consensus.cc:2376] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4. Re-sending same reply.
I20260709 13:15:46.101220 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:46.101614 11444 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.102203 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.102636 11442 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:46.103248 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:46.103713 11440 raft_consensus.cc:2376] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9. Re-sending same reply.
I20260709 13:15:46.087702 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.090145 14208 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.091706 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:46.095695 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.104749 11519 raft_consensus.cc:2410] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:15:46.097574 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:46.106372 11442 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:46.111557 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.112820 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.113355 11442 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:46.114003 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.114526 11442 raft_consensus.cc:2410] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:15:46.108206 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.085884 11262 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:46.118717 11437 raft_consensus.cc:2955] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:46.122481 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.123282 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:46.127871 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:46.128374 11445 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 3: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:46.133868 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.134544 11443 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 3.
I20260709 13:15:46.027169 14013 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:46.140697 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 215599us
I20260709 13:15:46.029912 14206 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:15:46.083204 14065 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:46.143651 12034 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:46.150219 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.136451 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:46.152782 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.143116 14021 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:46.144328 13891 consensus_meta.cc:287] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.842s	user 0.006s	sys 0.004s
I20260709 13:15:46.081460 14164 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:15:46.135669 14210 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:15:46.143817 14172 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.855s	user 0.010s	sys 0.000s
I20260709 13:15:46.169361 11444 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:46.014173 14200 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.022166 14184 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:46.178450 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 245701us
I20260709 13:15:46.139843 14014 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:46.051012 14027 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:15:46.181982 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:46.177983 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 159004us
I20260709 13:15:46.160498 13913 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.216796 11441 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:46.132799 14208 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:15:46.209722 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:46.153704 14182 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.139487 14012 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:15:46.199124 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 62750.9us
I20260709 13:15:46.235749 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.180806 14147 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:46.235008 11334 leader_election.cc:336] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.127s (SENT)
I20260709 13:15:46.176863 14021 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.237094 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.248593 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:46.249493 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:46.250185 14057 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:46.250545 14057 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.250854 14057 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:46.254724 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.255334 11511 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:15:46.190891 14210 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:46.225397 13873 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:46.221040 14014 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.290110 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:46.276156 14218 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:15:46.184445 13891 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.258133 14021 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:46.290343 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 484) took 1367 ms. Trace:
W20260709 13:15:46.315798 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 51614.2us
W20260709 13:15:46.315063 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 143296us
I20260709 13:15:46.335502 12034 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:46.335031 11439 rpcz_store.cc:276] 0709 13:15:44.923340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.264359 (+341019us) service_pool.cc:224] Handling call
0709 13:15:45.264637 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:45.264944 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:45.273495 (+  8551us) log.cc:844] Serialized 14 byte log entry
0709 13:15:45.573832 (+300337us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:45.686835 (+113003us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:45.687209 (+   374us) log.cc:844] Serialized 26 byte log entry
0709 13:15:45.687353 (+   144us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:45.687383 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.687457 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:46.289975 (+602518us) raft_consensus.cc:1685] finished
0709 13:15:46.290076 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.290276 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8402,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":1141,"raft.queue_time_us":6651,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":28211,"spinlock_wait_cycles":55355776,"thread_start_us":199647,"threads_started":1,"wal-append.queue_time_us":988229}
I20260709 13:15:46.283290 14220 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:46.284085 14224 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:46.282712 14193 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.283495 14207 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.222491 14172 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.346405 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.350456 11511 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:46.292239 14210 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:46.333927 14215 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:15:46.305765 14065 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:46.292093 14222 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:46.292549 14226 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.292970 14235 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.287150 14186 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:46.304288 14014 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:46.343144 14224 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.291769 14209 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:46.360906 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.301703 14188 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:46.364565 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 311623us
W20260709 13:15:46.364102 11333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.003s	sys 0.006s
I20260709 13:15:46.286909 14194 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.265s
I20260709 13:15:46.290970 14223 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:46.294098 14185 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.374238 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.362082 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.352039 14190 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:46.354326 14006 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:46.353842 14207 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.369647 14241 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:46.416230 14006 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:46.378750 14224 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:46.414851 14228 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
W20260709 13:15:46.393508 11333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.003s	sys 0.006s
W20260709 13:15:46.417838 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 723867us
I20260709 13:15:46.418745 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.378422 14222 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:46.419871 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.375747 13953 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:46.361155 14162 consensus_meta.cc:287] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.001s
I20260709 13:15:46.421187 11443 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:46.394320 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:46.359366 14008 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:46.362717 14065 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.425854 11368 raft_consensus.cc:606] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:46.426345 11368 raft_consensus.cc:614] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:46.426820 11370 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:46.320524 13790 raft_consensus.cc:2955] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:46.431838 14013 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.410808 14200 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.433547 14230 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.434589 14222 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.435464 14065 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:46.411859 13891 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.398800 14194 consensus_queue.cc:1243] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:46.416823 14006 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.427143 14241 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:46.448822 13848 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.887s	user 0.003s	sys 0.008s
W20260709 13:15:46.468827 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 74655us
I20260709 13:15:46.471653 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:46.449259 14192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.005s	sys 0.009s
I20260709 13:15:46.476747 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:46.466812 14241 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:46.449095 14093 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 13:15:46.475188 14192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.005s	sys 0.010s
I20260709 13:15:46.484406 14237 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:46.474372 14194 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.478523 14080 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:46.399569 14188 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.475394 13848 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.427886 14190 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.496274 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:46.497814 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 56616us
W20260709 13:15:46.500512 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 152095us
W20260709 13:15:46.505874 11334 connection.cc:256] RPC call timeout handler was delayed by -1.3961536419999447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
I20260709 13:15:46.485069 14241 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:46.514832 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:46.515772 11334 leader_election.cc:336] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.962s (SENT)
W20260709 13:15:46.516840 11334 leader_election.cc:336] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.055s (SENT)
W20260709 13:15:46.517876 14167 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
I20260709 13:15:46.518435 14167 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:46.520195 14082 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
I20260709 13:15:46.489319 14184 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.520470 14188 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.521162 14194 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:46.521579 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:46.517262 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:46.525226 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:46.525892 11334 leader_election.cc:336] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.151s (SENT)
I20260709 13:15:46.526903 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.527750 11513 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:46.530344 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.535982 12069 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:46.523576 14226 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.538085 13848 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:46.539101 14191 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:46.539779 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.540421 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:46.511595 14237 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:46.541069 12060 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:46.541693 14237 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.546654 14065 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.528445 13891 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:46.547746 13891 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.548102 13891 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:46.552666 14190 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.553591 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.556921 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 752) took 1002 ms. Trace:
I20260709 13:15:46.564842 13873 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.563903 12063 rpcz_store.cc:276] 0709 13:15:45.554134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.695988 (+141854us) service_pool.cc:224] Handling call
0709 13:15:45.753766 (+ 57778us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:45.892472 (+138706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.902023 (+  9551us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:46.054854 (+152831us) log.cc:844] Serialized 52 byte log entry
0709 13:15:46.254312 (+199458us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:46.480589 (+226277us) log.cc:844] Serialized 14 byte log entry
0709 13:15:46.490081 (+  9492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.497520 (+  7439us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:46.555549 (+ 58029us) raft_consensus.cc:1685] finished
0709 13:15:46.555889 (+   340us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.556834 (+   945us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3668,"prepare.queue_time_us":231769,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":4466,"raft.queue_time_us":13416,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":62474,"thread_start_us":226237,"threads_started":2,"wal-append.queue_time_us":365957}
I20260709 13:15:46.565331 13873 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.556562 14241 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.553651 14224 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.562189 12060 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
W20260709 13:15:46.569090 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 322166us
W20260709 13:15:46.570298 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45144.7us
I20260709 13:15:46.524920 14200 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:46.578984 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.579687 12063 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:46.573966 14184 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:46.516366 14057 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.593647 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:46.557423 14251 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:46.599749 12063 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:46.601455 14241 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.602265 14057 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.603008 11511 raft_consensus.cc:2368] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:46.603554 14224 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.594017 14184 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.607617 14184 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:46.571256 14006 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.628947 14258 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:46.631155 14258 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:46.634331 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.614610 14190 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:46.638427 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:46.639189 14172 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:46.641198 13917 consensus_meta.cc:287] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.617s	user 0.000s	sys 0.008s
I20260709 13:15:46.641671 13917 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.641799 13891 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.643657 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:46.644435 14014 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.646307 14006 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.639107 11361 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:46.646824 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 754) took 1092 ms. Trace:
I20260709 13:15:46.638784 11369 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:46.638504 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:46.644937 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.648339 11363 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:46.647200 12066 rpcz_store.cc:276] 0709 13:15:45.554627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.698629 (+144002us) service_pool.cc:224] Handling call
0709 13:15:45.757392 (+ 58763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:45.902699 (+145307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.646637 (+743938us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b54005212d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:46.646774 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134511488}
I20260709 13:15:46.632243 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.646507 14014 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:46.659957 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.661576 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:46.667589 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54831us
I20260709 13:15:46.668942 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:46.677490 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 105079us
I20260709 13:15:46.681295 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:46.683068 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.647611 14021 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.649518 14190 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:46.697301 14216 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.807s	user 0.012s	sys 0.000s
I20260709 13:15:46.663087 12061 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:46.665073 14260 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:46.698233 14256 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:15:46.682477 13917 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.704939 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.698428 14162 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:46.706943 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.708493 12060 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:46.709648 14162 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.710098 14162 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:46.713685 13873 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.702527 14065 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:46.716055 11441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 474) took 1894 ms (client timeout 1996 ms). Trace:
W20260709 13:15:46.716435 11441 rpcz_store.cc:269] 0709 13:15:44.821354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.851146 (+ 29792us) service_pool.cc:224] Handling call
0709 13:15:46.715892 (+1864746us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54003e3ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:46.716002 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1988414720}
I20260709 13:15:46.701930 13891 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.722647 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:46.723281 14252 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:15:46.724311 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.724982 12061 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:46.725633 14207 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.725873 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.726305 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:46.726970 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.727162 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.727288 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.727728 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.727924 12063 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:46.728295 12067 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:46.728165 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.703702 14187 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.706410 14260 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.729274 12069 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:46.733865 12060 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:15:46.739055 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.767343 11510 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:46.772495 12066 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:46.788920 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:46.733680 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 55197us
I20260709 13:15:46.790623 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.790984 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.791363 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.793017 11510 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
W20260709 13:15:46.784348 14262 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:15:46.791725 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.740972 14021 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.794405 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.792691 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.793185 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:46.747022 14171 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:46.748507 12061 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:46.797446 14265 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:46.799454 14251 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:46.800211 14265 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:46.801090 14265 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.801509 14162 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.800164 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 613) took 1083 ms. Trace:
I20260709 13:15:46.804418 11365 rpcz_store.cc:276] 0709 13:15:45.716583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.718256 (+  1673us) service_pool.cc:224] Handling call
0709 13:15:46.799452 (+1081196us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:46.799647 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:46.799826 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:46.799855 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:46.799895 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:46.799936 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.799991 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.800096 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6869760}
I20260709 13:15:46.797163 14171 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:46.807066 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.809894 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:46.718748 13873 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.791957 14264 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:46.741115 14184 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.810387 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.790333 14083 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.812155 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.831176 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.739787 14190 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:46.795373 11510 raft_consensus.cc:2410] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:15:46.808058 12060 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:46.823853 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:46.833107 11441 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:15:46.824642 13594 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:46.842470 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:46.843055 11437 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:46.843915 11443 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:46.841255 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:46.845042 11436 raft_consensus.cc:2368] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5.
I20260709 13:15:46.841987 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:46.847375 14171 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.848706 14171 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:46.852171 11408 leader_election.cc:336] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.528s (SENT)
I20260709 13:15:46.852618 11408 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:46.871060 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.871713 12068 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:46.828729 13891 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.852618 13953 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.867841 14083 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:46.881640 11437 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:46.861557 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.792172 14187 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.871006 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.828328 14210 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:46.876327 14162 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.884150 13891 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:46.893491 14277 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:46.885979 13953 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:46.981134 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.968510 11445 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:46.963383 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:46.993248 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:46.993398 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.994344 12065 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:46.995190 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.995668 12065 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:47.000555 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.004248 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.942952 12061 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:47.006078 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:46.990869 14266 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 13:15:47.000286 14275 log.cc:927] Time spent T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.073s	user 0.002s	sys 0.000s
I20260709 13:15:47.010353 11514 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:46.983062 14210 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:46.976949 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:46.939534 14260 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:46.999521 14193 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:47.020032 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:47.006491 14269 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.022104 11514 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:15:47.022505 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 83523us
I20260709 13:15:47.022984 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:47.025293 11366 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:47.030891 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:47.023123 11445 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:47.023226 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.033629 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.013397 14264 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:47.031529 11367 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:15:47.032339 12062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 740) took 1489 ms (client timeout 1665 ms). Trace:
W20260709 13:15:47.037201 12062 rpcz_store.cc:269] 0709 13:15:45.542925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.543108 (+   183us) service_pool.cc:224] Handling call
0709 13:15:47.032178 (+1489070us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54007783d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:47.032285 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931177216}
I20260709 13:15:47.038537 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:46.998402 14277 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.034158 11364 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:47.032526 11514 raft_consensus.cc:2410] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:47.024336 14184 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:47.051663 11366 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:47.052567 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.053247 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.057493 11443 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:47.058238 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:47.058691 11444 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:47.060534 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.035789 13891 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:47.066771 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:47.067327 11514 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:47.067986 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:47.028744 14269 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:47.050243 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.070609 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:47.079109 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:47.079878 11514 raft_consensus.cc:2410] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:15:47.061866 11443 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:47.077183 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.076524 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:47.083106 11519 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:47.069068 14264 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:47.089092 14273 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:15:47.098336 11512 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:47.082418 14276 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:15:47.104676 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1472) took 1017 ms. Trace:
I20260709 13:15:47.105137 11513 rpcz_store.cc:276] 0709 13:15:46.087058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:46.122236 (+ 35178us) service_pool.cc:224] Handling call
0709 13:15:47.104437 (+982201us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540014bed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:47.104602 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929751296}
I20260709 13:15:47.106594 11513 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:47.107227 11517 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:47.112010 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:47.114858 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:47.117821 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:47.082733 14187 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:47.134042 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:47.141331 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:47.146574 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:47.142256 11362 raft_consensus.cc:2410] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:15:47.144181 14088 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:47.140205 14098 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:47.157459 14269 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:47.145793 13953 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:47.176568 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:47.179446 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:47.185211 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.204391 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:47.205559 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.186005 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:47.186590 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.206439 12062 raft_consensus.cc:2376] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7. Re-sending same reply.
I20260709 13:15:47.206621 12065 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:47.151755 13661 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:47.152024 14247 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:47.208467 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.164142 14009 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:15:47.208755 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.224433 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:47.224804 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.225227 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.210896 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:47.226200 11366 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:47.154481 14193 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:15:47.212109 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.214136 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:47.214768 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.215647 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:47.215883 14285 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:15:47.216468 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.235092 12062 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:47.217396 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:47.234692 11510 raft_consensus.cc:2955] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:47.219992 14277 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:47.221714 14279 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:15:47.225620 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.246600 12065 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:15:47.210206 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:47.228183 11409 connection.cc:256] RPC call timeout handler was delayed by -1.0873980909998409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:15:47.247735 11409 connection.cc:256] RPC call timeout handler was delayed by -1.0522980909997841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
I20260709 13:15:47.236541 12069 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:47.237356 12061 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:47.238066 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.257265 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:47.258805 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.238374 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.261798 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.236296 14207 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:15:47.244136 12064 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:47.234292 14169 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.244712 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.252879 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.269625 12065 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:47.264225 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.272950 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:47.278359 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.279070 11367 raft_consensus.cc:2393] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:47.279624 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:47.260042 13891 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:47.300246 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.301759 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.279933 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.322448 11366 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
W20260709 13:15:47.654137 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 395357us
I20260709 13:15:47.654556 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.660562 11369 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:15:47.643654 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1461) took 1615 ms. Trace:
I20260709 13:15:47.664529 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.664820 11511 rpcz_store.cc:276] 0709 13:15:46.028239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:46.802170 (+773931us) service_pool.cc:224] Handling call
0709 13:15:46.802456 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:46.802632 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:46.885690 (+ 83058us) log.cc:844] Serialized 14 byte log entry
0709 13:15:47.643245 (+757555us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:47.643308 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:47.643361 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:47.643448 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.643588 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3018,"prepare.queue_time_us":222199,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":53,"thread_start_us":459717,"threads_started":2,"wal-append.queue_time_us":637415}
I20260709 13:15:47.668569 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.673817 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.675518 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.667058 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.675657 14083 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:47.272106 14169 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:47.677580 14120 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:47.677845 14021 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:47.683651 14120 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:47.683569 13891 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.684670 14120 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.686106 14120 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:47.685729 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.266337 14223 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:47.687415 12065 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 13:15:47.678666 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 416918us
I20260709 13:15:47.689332 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:47.689299 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.690724 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:47.690991 11366 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:15:47.693029 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.693688 11363 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:47.360347 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:47.678504 11514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.000s	sys 0.008s
W20260709 13:15:47.615513 13989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.015s	sys 0.011s
W20260709 13:15:47.405808 14199 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.521s	user 0.005s	sys 0.000s
W20260709 13:15:47.697103 11514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.001s	sys 0.008s
I20260709 13:15:47.698128 12068 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:15:47.292543 14184 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:47.703544 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 262184us
I20260709 13:15:47.700750 12066 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:47.701846 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:47.702699 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.350703 13594 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:47.495604 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.405645 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:47.280838 14200 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:47.692485 14290 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:15:47.704809 11369 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:47.705607 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1887) took 1006 ms. Trace:
I20260709 13:15:47.706910 12065 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:47.686848 13891 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:47.700399 13989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.015s	sys 0.014s
I20260709 13:15:47.757580 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:47.756778 14084 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.696846 14223 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:47.702855 14207 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:15:47.697690 14171 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:47.750975 11514 rpcz_store.cc:276] 0709 13:15:46.698773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.080343 (+381570us) service_pool.cc:224] Handling call
0709 13:15:47.080578 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:47.080718 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:47.115520 (+ 34802us) log.cc:844] Serialized 14 byte log entry
0709 13:15:47.705174 (+589654us) log.cc:844] Serialized 14 byte log entry
0709 13:15:47.705320 (+   146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:47.705357 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:47.705393 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:47.705454 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.705557 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26401,"prepare.queue_time_us":29875,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":9756,"raft.queue_time_us":8150,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":7324,"spinlock_wait_cycles":114139776,"thread_start_us":584375,"threads_started":1}
I20260709 13:15:47.703514 14184 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:47.774480 14171 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:47.782011 14171 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:47.782552 14293 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:15:47.783914 14296 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:15:47.709764 14228 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:15:47.769361 14223 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:47.781869 14294 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:15:47.750479 14018 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:15:47.785777 14184 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:47.792701 14291 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 13:15:47.809765 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:47.810994 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:47.812127 11335 leader_election.cc:400] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5.
I20260709 13:15:47.795918 14223 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:47.813122 11519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.038s	sys 0.027s
I20260709 13:15:47.772333 14084 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:47.814242 11519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.039s	sys 0.027s
I20260709 13:15:47.814173 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:47.814805 14258 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:47.818574 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:47.816776 14084 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:47.819641 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 155207us
I20260709 13:15:47.823367 11445 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:47.814114 14228 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:47.821264 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.829087 11437 raft_consensus.cc:1217] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-10.9]   Dedup: 6.7->[8.8-10.9]
I20260709 13:15:47.837988 11437 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: [6.7-10.9]
I20260709 13:15:47.831537 12065 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:47.842105 11438 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:15:47.844785 12034 connection.cc:256] RPC call timeout handler was delayed by -1.2288865300001817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1996s for a response
I20260709 13:15:47.844899 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.845435 11437 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:47.846411 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:47.819859 14230 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:47.849545 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:47.818261 14258 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.756479 13594 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.853226 14009 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.852291 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.870745 11436 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:47.850659 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:47.893147 14234 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:47.893464 14227 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:47.893965 14227 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:47.855682 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.875063 14258 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:47.875844 14009 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.855115 13989 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:47.821740 14169 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:47.880833 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:47.865506 14301 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:15:47.896806 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:47.912890 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.899577 11437 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:47.847697 11365 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:47.905985 11367 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:47.825070 14065 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:15:47.843196 14084 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:47.867537 11361 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:47.896121 14303 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:47.918948 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.920491 13891 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:47.922458 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 222109us
I20260709 13:15:47.853178 14305 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:47.896386 14230 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:47.927716 11436 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:47.894855 14227 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.931969 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:47.921929 14297 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.103s	user 0.001s	sys 0.002s
I20260709 13:15:47.930524 13844 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:15:47.935621 14227 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:47.905377 13913 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:47.937676 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:47.938740 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:47.939566 14308 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:47.935379 14230 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.940079 14308 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:47.941001 14308 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:47.941869 14302 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:15:47.928524 14171 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:47.943590 14120 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:47.944037 14120 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.944396 14120 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:47.907579 13989 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:47.946537 14207 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:47.905449 14193 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:15:47.921272 14084 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:47.954813 11363 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:47.952142 14305 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:47.947844 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:47.944281 14171 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:47.928501 14303 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:47.948374 14187 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:47.991788 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 43991.9us
I20260709 13:15:47.964910 11437 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:47.993872 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:47.995038 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:47.996507 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.997902 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.988066 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:47.989434 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 92121.7us
I20260709 13:15:47.950060 13989 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:47.965168 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.996452 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:47.947249 14083 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:47.931588 13891 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:47.963704 14084 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:48.042423 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:48.052392 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 136545us
I20260709 13:15:48.053224 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 589) took 1155 ms. Trace:
I20260709 13:15:48.053617 11518 rpcz_store.cc:276] 0709 13:15:46.897563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.042125 (+1144562us) service_pool.cc:224] Handling call
0709 13:15:48.053141 (+ 11016us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:47.991524 14303 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.011556 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.059593 12065 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:48.060633 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:48.012430 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.014721 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.015833 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.016106 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.061650 12064 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:48.016440 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:48.021130 14234 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:48.004302 14227 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.062436 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.004230 14169 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:48.062166 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:48.062610 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:48.063165 12066 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:48.063472 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.063764 14234 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.064497 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:48.061645 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.040323 13891 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:48.065905 12063 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:48.062281 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:47.991835 14305 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:48.072500 12066 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:48.064138 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:48.067277 12068 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:48.068013 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:48.050117 14062 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:48.070555 13661 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:48.073499 14186 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:48.074846 14193 consensus_queue.cc:1243] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:48.062778 14227 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:48.090194 14186 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:48.089721 14187 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:48.096323 14186 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:48.126492 14269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.003s	sys 0.018s
W20260709 13:15:48.333791 14269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.117s	user 0.005s	sys 0.018s
I20260709 13:15:48.334254 14305 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:48.188684 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 369904us
W20260709 13:15:48.204797 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 275179us
I20260709 13:15:48.336885 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:47.999301 11437 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:48.340670 14057 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.341665 11437 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:48.342021 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1614) took 1114 ms. Trace:
I20260709 13:15:48.342473 11367 rpcz_store.cc:276] 0709 13:15:47.227342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.279723 (+ 52381us) service_pool.cc:224] Handling call
0709 13:15:48.341579 (+1061856us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54000a73d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:48.341775 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551172992}
I20260709 13:15:48.031142 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:48.344269 11443 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:48.344893 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:48.343719 11366 raft_consensus.cc:2955] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:47.992678 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:48.346736 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 354008us
I20260709 13:15:48.345638 11369 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.081453 13594 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.351387 14308 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:48.352066 14308 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:48.352524 14308 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:48.353585 11367 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:48.079392 14062 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:48.361773 14062 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:48.295845 14186 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:48.341588 14057 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.254050 12064 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:48.297626 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.301048 14009 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:48.073200 12062 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:48.368577 13989 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.371729 11442 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:48.301587 13891 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.304445 11362 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:48.377051 11518 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:15:48.304947 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:48.381775 14167 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:48.382231 14167 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:48.384299 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 48193.8us
W20260709 13:15:48.384964 14184 consensus_meta.cc:287] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.573s	user 0.001s	sys 0.016s
I20260709 13:15:48.385509 14184 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.308588 14084 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.309949 14065 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:48.310107 14200 consensus_meta.cc:287] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.005s
I20260709 13:15:48.318377 14227 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:48.089509 14314 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.087697 13661 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:48.336076 14318 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:48.345703 11443 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:48.373811 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.383355 14167 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.391335 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:48.393515 13891 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.373060 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:48.394943 11369 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:48.371125 13989 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.378399 14258 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:48.395291 14284 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:48.414398 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:48.415913 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 79189.8us
I20260709 13:15:48.409371 14318 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:48.414089 14284 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:48.414222 14057 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.431689 14227 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.433838 14184 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.434059 14167 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:48.434286 14314 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.436579 14187 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:48.437077 14187 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:48.437184 14314 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:48.437528 14187 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:48.440056 13661 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.441289 13661 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:48.427740 14065 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.427525 14200 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:48.448053 11440 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:48.448338 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.449347 11515 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:15:48.450169 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.450736 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.451411 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.452263 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.453016 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.453055 11515 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:48.453909 11512 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:15:48.455361 14284 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:48.456774 14284 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:48.459604 11364 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:48.451546 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.433838 11519 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:15:48.463964 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 72823.2us
I20260709 13:15:48.468242 14091 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:48.468604 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2159) took 1239 ms. Trace:
I20260709 13:15:48.468858 14091 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:48.469019 11365 rpcz_store.cc:276] 0709 13:15:47.229511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.757047 (+527536us) service_pool.cc:224] Handling call
0709 13:15:47.767231 (+ 10184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:47.918260 (+151029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:48.465625 (+547365us) spinlock_profiling.cc:243] Waited 46.9 ms on lock 0x7b54000f91d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:48.468418 (+  2793us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107959168}
I20260709 13:15:48.470498 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.477785 14317 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.478745 14317 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.482213 11439 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:48.478273 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 106836us
I20260709 13:15:48.480175 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:48.489976 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:48.495455 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.497567 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:48.497774 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:48.498884 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.500360 11515 raft_consensus.cc:2368] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:48.501250 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.499684 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.503329 11510 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:48.505139 11514 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:15:48.504452 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 152959us
I20260709 13:15:48.499223 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:48.507683 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:48.508745 14171 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:48.509347 14171 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.509876 14171 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:48.499799 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.513375 11517 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:48.514006 14091 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.520128 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:48.522186 11511 raft_consensus.cc:2393] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:48.514211 14303 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.523859 11442 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:48.497437 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:48.526326 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.529596 14018 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:48.531903 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:48.529801 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50722 (request call id 23) took 3443 ms. Trace:
I20260709 13:15:48.532696 11365 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:48.541867 11368 rpcz_store.cc:276] 0709 13:15:45.086120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.086290 (+   170us) service_pool.cc:224] Handling call
0709 13:15:48.529374 (+3443084us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001e64d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f8853 00007f35c19c660a 00007f35baf4c7d4 00007f35baf4c762 00007f35baf4c691 00007f35baf4c60c 00007f35baf4b8c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:48.529523 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2804085504}
I20260709 13:15:48.545851 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:48.547510 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:48.549660 13953 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:48.551761 14120 consensus_meta.cc:287] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.607s	user 0.019s	sys 0.002s
I20260709 13:15:48.552299 14120 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.554831 14120 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.556118 11513 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:48.550069 14330 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:15:48.549525 14303 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:48.555989 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 567901us
I20260709 13:15:48.545348 14331 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:48.559985 14084 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.560052 14331 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:48.560736 14331 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:48.560976 14241 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:48.561602 14241 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.563956 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:48.564651 11439 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:48.565886 14317 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.568372 11368 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:48.568254 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:48.550043 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:48.579623 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 441215us
I20260709 13:15:48.580744 12034 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:48.582798 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:48.583293 14171 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:48.584154 11437 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:15:48.555892 14223 consensus_meta.cc:287] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.737s	user 0.000s	sys 0.018s
W20260709 13:15:48.585649 11370 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 609) took 2870 ms (client timeout 2205 ms). Trace:
W20260709 13:15:48.589687 11370 rpcz_store.cc:269] 0709 13:15:45.715437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.716785 (+  1348us) service_pool.cc:224] Handling call
0709 13:15:48.585291 (+2868506us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54001e64d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:48.585453 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545278848}
I20260709 13:15:48.608881 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.610301 11369 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:48.608777 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:48.611304 11365 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:15:48.612591 11512 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:15:48.586232 14084 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:15:48.616827 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45467.6us
I20260709 13:15:48.619509 11517 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:48.610209 14171 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:48.625026 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 105015us
I20260709 13:15:48.504452 14318 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.590129 14223 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:48.638540 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 88670.3us
I20260709 13:15:48.641192 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:48.642838 14333 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:15:48.643663 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63144.1us
I20260709 13:15:48.643780 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.647495 11510 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:48.648828 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.649475 11510 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:48.648931 14223 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:48.646518 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:48.620803 14303 raft_consensus.cc:2764] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:15:48.657792 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:48.766600 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 129931us
I20260709 13:15:48.749235 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:48.623122 14187 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:48.626847 14084 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:48.803737 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.624294 14308 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:48.813817 14299 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.667s	user 0.003s	sys 0.001s
I20260709 13:15:48.706504 14317 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:15:49.149427 14241 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.152310 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2275) took 1188 ms. Trace:
I20260709 13:15:49.153123 11439 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
W20260709 13:15:49.169848 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.153491 11518 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:49.154273 14227 consensus_meta.cc:287] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.010s
I20260709 13:15:49.157096 14200 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:48.765816 14330 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:49.182623 14330 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:49.183094 14330 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:49.184801 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:49.188844 14009 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:49.191505 14308 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:49.192502 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 546080us
I20260709 13:15:49.196488 11517 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:49.198158 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:49.178171 14227 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:49.206163 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47726.8us
I20260709 13:15:49.160391 11513 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:48.648969 13844 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.141s
I20260709 13:15:49.158490 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.174490 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:15:49.180897 11514 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.674s	user 0.017s	sys 0.000s
I20260709 13:15:49.169644 11436 rpcz_store.cc:276] 0709 13:15:47.963754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.965934 (+  2180us) service_pool.cc:224] Handling call
0709 13:15:47.966257 (+   323us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:48.394904 (+428647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:48.394950 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:48.395887 (+   937us) log.cc:844] Serialized 78 byte log entry
0709 13:15:48.628943 (+233056us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:48.628979 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:48.636085 (+  7106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:49.151927 (+515842us) raft_consensus.cc:1685] finished
0709 13:15:49.151946 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.152130 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108844,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":2130,"thread_start_us":632914,"threads_started":2,"wal-append.queue_time_us":386681}
W20260709 13:15:49.213440 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 559121us
W20260709 13:15:49.214327 14284 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.757s	user 0.011s	sys 0.011s
I20260709 13:15:49.215104 14171 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:49.217818 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.218766 14171 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:49.215299 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:49.220222 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.221064 14018 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:49.221925 14018 consensus_queue.cc:1243] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:49.220331 14171 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:49.175256 14331 consensus_meta.cc:287] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.013s
I20260709 13:15:49.230173 11510 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:49.176210 11408 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.727s (SENT) [suppressed 38 similar messages]
W20260709 13:15:49.233297 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.233808 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 773) took 1211 ms. Trace:
W20260709 13:15:49.179885 14167 consensus_meta.cc:287] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.014s
I20260709 13:15:49.164819 14333 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:49.237443 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 55585us
I20260709 13:15:49.171131 13953 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:49.238948 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.241955 11519 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:49.243434 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.244232 11511 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:49.236011 12062 rpcz_store.cc:276] 0709 13:15:48.018587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.072564 (+ 53977us) service_pool.cc:224] Handling call
0709 13:15:48.072818 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:48.073116 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:48.415452 (+342336us) log.cc:844] Serialized 14 byte log entry
0709 13:15:48.624736 (+209284us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:48.638651 (+ 13915us) log.cc:844] Serialized 26 byte log entry
0709 13:15:48.638875 (+   224us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:49.064317 (+425442us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.205958 (+141641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.230029 (+ 24071us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:49.230074 (+    45us) raft_consensus.cc:1685] finished
0709 13:15:49.230087 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.230230 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58861,"raft.queue_time_us":286115,"raft.run_cpu_time_us":603,"raft.run_wall_time_us":11120,"thread_start_us":601933,"threads_started":2,"wal-append.queue_time_us":222545}
I20260709 13:15:49.259037 14223 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:49.259482 11515 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:49.260830 13953 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:49.261229 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.262848 14247 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:49.263387 14247 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:49.264539 14247 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.265666 14247 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:49.265889 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2013) took 2113 ms. Trace:
I20260709 13:15:49.266280 11443 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:15:49.267800 11442 raft_consensus.cc:1217] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-10.9]   Dedup: 6.7->[8.8-10.9]
I20260709 13:15:49.268524 11442 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: [6.7-10.9]
W20260709 13:15:49.269173 13661 consensus_meta.cc:287] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.008s
W20260709 13:15:49.269671 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 108773us
I20260709 13:15:49.243831 14331 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.271626 13891 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:49.272118 13891 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:49.272487 13891 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:49.272828 14331 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.274114 11511 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:49.274350 14084 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.278537 14308 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:49.255100 11516 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:49.259203 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:49.291831 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.255360 14187 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:49.314527 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 62364.9us
I20260709 13:15:49.247131 14284 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
W20260709 13:15:49.319299 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47922.7us
I20260709 13:15:49.266294 11444 rpcz_store.cc:276] 0709 13:15:47.152715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.152972 (+   257us) service_pool.cc:224] Handling call
0709 13:15:47.153416 (+   444us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:47.158276 (+  4860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:47.158313 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:47.159099 (+   786us) log.cc:844] Serialized 52 byte log entry
0709 13:15:48.519809 (+1360710us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b1800492be0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:15:49.156674 (+636865us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:49.156799 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.157039 (+   240us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:49.265445 (+108406us) raft_consensus.cc:1685] finished
0709 13:15:49.265467 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.265703 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":597,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":9504,"spinlock_wait_cycles":11533056,"thread_start_us":132039,"threads_started":1,"wal-append.queue_time_us":2103113}
I20260709 13:15:49.290990 14167 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:49.280049 14333 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.328749 11511 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:15:49.328701 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.321887 11335 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:49.312670 13844 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:49.310983 13661 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:49.326102 14171 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:49.343662 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 552964us
I20260709 13:15:49.325249 14303 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:49.331455 14187 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:49.352991 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56490us
I20260709 13:15:49.353849 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.346437 14227 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:49.341075 14167 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.367964 14213 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:49.368300 14167 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.368891 13913 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:49.369292 13913 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:49.369787 14171 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.370327 13913 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.371650 14184 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:49.369238 14080 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:49.352192 13661 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.350909 13891 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:49.385375 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 124244us
W20260709 13:15:49.391006 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51877.5us
I20260709 13:15:49.378911 13913 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:49.372718 14187 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.362504 14241 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:15:49.373366 14227 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:49.354422 13844 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:49.398259 11409 connection.cc:256] RPC call timeout handler was delayed by -1.6973225740000544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
I20260709 13:15:49.381810 14303 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.369298 13953 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.382331 14284 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:49.436983 11409 connection.cc:256] RPC call timeout handler was delayed by -1.6361225739999554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
W20260709 13:15:49.477769 11409 connection.cc:256] RPC call timeout handler was delayed by -1.5272225740000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
W20260709 13:15:49.478137 11409 connection.cc:256] RPC call timeout handler was delayed by -1.3742225739999867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2378s for a response
W20260709 13:15:49.478526 11409 connection.cc:256] RPC call timeout handler was delayed by -1.3526225739999518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24020000000000002s for a response
W20260709 13:15:49.419917 14342 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:15:49.483072 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54405.7us
I20260709 13:15:49.425747 14187 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.428722 14084 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.385267 14080 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.395321 14351 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:15:49.488382 14303 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.489810 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:49.489591 14330 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:49.490720 13661 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:49.491640 14353 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:49.424556 14352 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:15:49.491315 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1894) took 1011 ms. Trace:
I20260709 13:15:49.490710 14348 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.493637 14227 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:49.498569 11334 connection.cc:256] RPC call timeout handler was delayed by -1.3946699000000535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 13:15:49.500005 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.500519 13953 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:49.489617 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 85567.8us
I20260709 13:15:49.419013 14251 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:49.479002 14284 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.492274 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 622) took 1650 ms. Trace:
I20260709 13:15:49.528468 11514 rpcz_store.cc:276] 0709 13:15:48.480219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.480345 (+   126us) service_pool.cc:224] Handling call
0709 13:15:48.485168 (+  4823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.504842 (+ 19674us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:49.209801 (+704959us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.485085 (+275284us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.490898 (+  5813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:49.490955 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:49.491002 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.491076 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.491233 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5556,"prepare.queue_time_us":7422,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":84658,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":39165,"spinlock_wait_cycles":46099200,"thread_start_us":96545,"threads_started":1,"wal-append.queue_time_us":242136}
W20260709 13:15:49.537681 11334 connection.cc:256] RPC call timeout handler was delayed by -1.3878321109998524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
I20260709 13:15:49.539369 14284 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:49.541168 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.540736 11436 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:49.541903 14353 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:49.542876 14080 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.543195 14247 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:15:49.543910 13913 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:49.546937 14348 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.536243 11364 rpcz_store.cc:276] 0709 13:15:47.841914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.935107 (+ 93193us) service_pool.cc:224] Handling call
0709 13:15:47.935467 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.465672 (+530205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.434696 (+969024us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54000f96d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:49.492194 (+ 57498us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092267392}
I20260709 13:15:49.522662 14084 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.527832 14213 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:15:49.553916 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:49.553001 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 376824us
W20260709 13:15:49.548568 11334 connection.cc:256] RPC call timeout handler was delayed by -1.2168321110000306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 13:15:49.563606 11334 connection.cc:256] RPC call timeout handler was delayed by -1.1619321109999419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
W20260709 13:15:49.563980 11334 connection.cc:256] RPC call timeout handler was delayed by -1.1376321109999026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 13:15:49.564239 11334 connection.cc:256] RPC call timeout handler was delayed by -1.1192525040000874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 13:15:49.564497 11334 connection.cc:256] RPC call timeout handler was delayed by -1.1190228650000336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
W20260709 13:15:49.564775 11334 connection.cc:256] RPC call timeout handler was delayed by -1.0584321110000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 13:15:49.565040 11334 connection.cc:256] RPC call timeout handler was delayed by -1.0377321109999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260709 13:15:49.540795 14346 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.536605 13661 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.536053 14251 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:49.578478 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 193469us
I20260709 13:15:49.580297 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:49.581789 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.582778 14359 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:49.583385 14359 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.585768 14359 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:49.589866 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:49.599041 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:49.599211 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:15:49.601890 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 99911.8us
I20260709 13:15:49.604460 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:49.605805 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 210235us
I20260709 13:15:49.572549 14213 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.592845 14187 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:49.618352 14303 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:49.618752 14303 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:49.619093 14303 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:49.556486 13953 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.534507 11437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:15:49.625741 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:49.627851 14348 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.628269 11436 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:49.628664 14348 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:49.629331 14227 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:49.630240 14303 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.632248 14303 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.623984 14346 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:49.633628 14334 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.147s	user 0.020s	sys 0.006s
W20260709 13:15:49.646546 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47436.1us
I20260709 13:15:49.626531 11437 raft_consensus.cc:606] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:49.593278 13661 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.614259 14213 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:49.613737 13891 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:49.652640 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 118434us
I20260709 13:15:49.574156 13913 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.646035 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:49.623657 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:49.658181 11408 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.954s (SENT)
I20260709 13:15:49.662798 11408 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:49.665419 11437 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:49.666461 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:49.666767 11445 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:49.667083 12069 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:49.668408 13953 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.668907 13953 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:49.673764 14318 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:49.674693 11408 leader_election.cc:336] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.115s (SENT)
I20260709 13:15:49.675158 11408 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.663060 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:49.683300 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.682327 12069 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
W20260709 13:15:49.685000 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59341.6us
I20260709 13:15:49.685880 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:49.686107 11440 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:49.687009 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:49.675536 14284 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:49.688144 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:49.674319 14251 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:49.689111 14251 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:49.689397 14284 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.690006 14284 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:49.690361 14284 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:49.690699 14284 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:49.687180 12064 raft_consensus.cc:2368] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:49.686146 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:49.692734 12061 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:49.687866 12034 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:49.662230 13844 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.697257 12033 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:49.700160 13891 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:49.700773 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.705041 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:49.705796 11364 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:49.709066 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:49.711992 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:49.712774 12068 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:49.713114 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.715914 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:49.717133 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:49.717743 12065 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:49.676883 14346 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.718832 14357 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:49.720152 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:49.720290 13953 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.720796 12063 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:49.720844 13953 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.722872 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1901) took 1080 ms. Trace:
I20260709 13:15:49.723047 13953 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.723244 11517 rpcz_store.cc:276] 0709 13:15:48.642685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.642860 (+   175us) service_pool.cc:224] Handling call
0709 13:15:48.643236 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.196908 (+553672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.722623 (+525715us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54002a76d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:49.722782 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1271906688}
I20260709 13:15:49.724383 13661 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:49.725185 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:49.725334 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:49.725989 11518 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:49.726090 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2021) took 1147 ms. Trace:
W20260709 13:15:49.720808 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54542.6us
W20260709 13:15:49.727519 11408 leader_election.cc:336] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.174s (SENT)
I20260709 13:15:49.727871 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:49.730391 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:49.729307 11408 leader_election.cc:336] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.228s (SENT)
W20260709 13:15:49.732781 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 69520us
I20260709 13:15:49.733142 11408 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:49.733098 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 59278.3us
I20260709 13:15:49.719246 14357 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.726473 11439 rpcz_store.cc:276] 0709 13:15:48.578515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.592761 (+ 14246us) service_pool.cc:224] Handling call
0709 13:15:48.620345 (+ 27584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.170046 (+549701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.725889 (+555843us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b54003e4ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:49.726037 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224923392}
I20260709 13:15:49.718779 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:49.712684 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:49.742825 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:49.743614 11517 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:49.742486 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:49.767882 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 103274us
I20260709 13:15:49.732555 14330 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.692098 14363 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:15:49.732790 13953 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:49.738201 14357 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:49.800899 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 244556us
W20260709 13:15:49.784087 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 98294.3us
I20260709 13:15:49.798944 13891 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:15:49.799484 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 102302us
I20260709 13:15:49.742554 13844 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:49.762324 14187 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:49.767705 14351 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:49.810520 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 620) took 1990 ms. Trace:
I20260709 13:15:49.767972 13913 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:49.771912 13661 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:49.810900 11366 rpcz_store.cc:276] 0709 13:15:47.820549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.820734 (+   185us) service_pool.cc:224] Handling call
0709 13:15:48.342677 (+521943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.343198 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:48.467050 (+123852us) log.cc:844] Serialized 14 byte log entry
0709 13:15:48.748841 (+281791us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.156105 (+407264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:49.156147 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:49.156199 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.156292 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.810365 (+654073us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54000a73d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:49.810467 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111532,"prepare.run_cpu_time_us":13227,"prepare.run_wall_time_us":376795,"raft.queue_time_us":7886,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":15205,"spinlock_wait_cycles":1137300352,"thread_start_us":140777,"threads_started":1,"wal-append.queue_time_us":913587}
I20260709 13:15:49.742323 14359 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:15:49.825940 12033 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:49.831558 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 102271us
I20260709 13:15:49.804634 13891 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:49.809973 14351 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.835505 13661 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:49.835997 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:49.836591 13661 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.834949 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:49.799010 14365 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:15:49.813020 14084 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:49.856465 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:49.857503 11409 leader_election.cc:400] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:49.831619 13844 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.839005 14351 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:49.857991 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:49.834481 13891 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:49.860298 14330 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:49.879196 12061 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:49.863261 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:49.866425 11408 leader_election.cc:336] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.796s (SENT)
I20260709 13:15:49.867341 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:49.868026 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:49.868532 12066 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:49.890699 12065 raft_consensus.cc:2410] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:49.869192 11409 leader_election.cc:403] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:15:49.878890 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47224.7us
I20260709 13:15:49.862490 12063 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:49.904680 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:49.905048 14359 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:49.905506 14359 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:49.905944 14359 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:49.906623 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.907300 14364 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:49.904711 12064 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:49.907948 14364 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:49.909057 14364 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.889878 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.912417 14330 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:49.913621 14330 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.906097 13844 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:49.907011 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:49.941395 14171 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:49.914387 14284 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:49.943351 14284 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:49.947480 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:49.948884 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.949579 11437 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:49.949733 14314 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:49.950307 14314 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:49.950977 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 116092us
W20260709 13:15:49.951617 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 85219.6us
I20260709 13:15:49.862534 14213 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:49.953132 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:49.958281 11442 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:49.954125 11436 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:49.956544 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 99195.7us
I20260709 13:15:49.963804 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:49.965147 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:49.971076 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:49.974840 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:49.975864 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:49.951819 14314 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:49.977289 11408 leader_election.cc:336] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.945s (SENT)
I20260709 13:15:49.981468 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:49.966780 14080 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.983464 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:49.984287 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:49.988871 11516 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:49.961539 14306 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 4. Current term is 5.
I20260709 13:15:49.972884 13953 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:49.991379 11518 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:15:49.992894 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 254075us
I20260709 13:15:49.950585 14351 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:49.997175 11442 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:52046
I20260709 13:15:49.997790 11442 raft_consensus.cc:606] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:49.998163 11442 raft_consensus.cc:614] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:15:49.999933 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.979957 14303 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:15:50.012571 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 105705us
I20260709 13:15:49.990072 14317 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:49.994477 13953 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.016916 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.017431 13953 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.022696 14080 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.024264 14357 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.025550 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.026964 14357 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.029702 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.030254 11510 raft_consensus.cc:2376] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5. Re-sending same reply.
I20260709 13:15:50.030748 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.031754 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.032292 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.033354 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.034621 12064 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:50.036404 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.032441 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 497) took 2204 ms. Trace:
I20260709 13:15:50.042296 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:50.061244 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46345us
I20260709 13:15:50.051251 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:50.032476 11513 raft_consensus.cc:2393] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:50.064128 11516 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:50.051581 14359 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.014272 14314 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.024876 14080 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.076702 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.082990 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.060532 11441 rpcz_store.cc:276] 0709 13:15:47.827727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.827854 (+   127us) service_pool.cc:224] Handling call
0709 13:15:47.851224 (+ 23370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:47.952921 (+101697us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:48.224867 (+271946us) log.cc:844] Serialized 14 byte log entry
0709 13:15:48.428144 (+203277us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:48.493475 (+ 65331us) log.cc:844] Serialized 26 byte log entry
0709 13:15:48.493642 (+   167us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:48.493676 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.632931 (+1139255us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:50.032045 (+399114us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400315bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:50.032097 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:50.032138 (+    41us) raft_consensus.cc:1685] finished
0709 13:15:50.032153 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.032303 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25958,"prepare.queue_time_us":123572,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":149849,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":38915,"spinlock_wait_cycles":236577920,"thread_start_us":327267,"threads_started":2,"wal-append.queue_time_us":162579}
I20260709 13:15:50.083717 11365 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:49.994891 14333 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:15:50.089745 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.090819 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.092111 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:50.092957 14332 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:50.093403 14332 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:50.094449 14332 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:50.095443 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.096592 14333 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.097141 14379 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:50.112273 14333 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.098070 14365 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.100605 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.102231 11439 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:50.068377 14167 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.109643 11516 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:15:50.112643 14379 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:50.099130 14382 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.082279 14171 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.126232 14171 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.108877 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:50.110936 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 58676.1us
I20260709 13:15:50.117136 11443 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:50.086798 14187 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:50.118918 11408 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:50.078627 14223 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:15:50.121153 14382 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.122817 14065 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:50.122615 14365 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.123971 14167 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.081904 14368 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:15:50.125252 14379 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.142769 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:50.148830 14330 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:50.161505 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:50.166250 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:50.169617 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:50.170598 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 60688.1us
I20260709 13:15:50.179895 11408 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.181680 11408 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.186386 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:50.172811 14351 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.176849 14379 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.189393 11439 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:50.201766 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:50.202239 11439 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:50.190094 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:50.193635 14382 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:50.194159 14333 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.184141 14314 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:15:50.205583 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.200486 13891 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.197688 11334 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:50.207964 13891 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.208559 13891 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:50.211606 11437 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:50.214152 14284 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:50.214614 14284 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.214937 11334 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.215987 14167 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:50.217293 11408 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:50.217927 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:50.218165 14365 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.218688 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.172346 14317 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:50.172652 14303 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:50.159024 14330 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.204192 14359 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.204406 14171 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:50.266311 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:50.214985 14284 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:50.198261 14368 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:50.223683 11436 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:50.199290 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:50.214144 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:50.203553 14083 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:50.271790 14083 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.272197 14083 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:50.272409 13953 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.274614 13953 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:50.234793 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:50.275418 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.275905 11365 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:15:50.235718 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45702.6us
I20260709 13:15:50.186012 14364 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:50.170490 14377 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:15:50.205732 14314 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:50.189107 14332 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.267019 11368 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:50.205904 14258 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.269403 11437 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:50.277653 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:50.283267 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 77779.4us
W20260709 13:15:50.280792 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 64163.3us
W20260709 13:15:50.281328 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 62698.4us
I20260709 13:15:50.282058 13953 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:50.302927 11368 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:50.303974 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.287922 14362 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:50.289583 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.280401 14314 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:50.275571 13844 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.291788 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.292466 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.295424 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:50.295756 14317 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.296438 14351 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:50.296787 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:50.283492 14187 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:50.316555 14187 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:50.317775 14368 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:50.297549 11444 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:50.187788 14386 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:15:50.318424 13844 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.269397 14167 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:50.300647 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:50.275532 14378 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.268867 14382 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:50.266161 13913 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:50.301781 11439 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:50.306805 11368 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:50.262225 14330 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:50.308449 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:50.313918 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:50.315790 14359 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:50.315963 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.320714 13844 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:50.319845 12061 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.324712 13953 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:50.320499 14372 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:50.322769 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.306109 14362 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:50.358371 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.359000 11518 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:50.329206 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.334326 11365 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:50.336396 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:15:50.337493 14387 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:15:50.338450 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:50.364429 12034 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:50.324549 11436 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:50.349100 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.362967 12033 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:50.344980 14223 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:50.395684 11512 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:50.440658 11366 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
W20260709 13:15:50.457237 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 94260.2us
W20260709 13:15:50.410957 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 95065.2us
W20260709 13:15:50.389642 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 75798.8us
I20260709 13:15:50.407517 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.366971 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.460389 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:50.410615 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:50.412087 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 111498us
W20260709 13:15:50.424629 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 55161.6us
I20260709 13:15:50.437364 12061 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:50.463764 11511 raft_consensus.cc:2368] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7.
I20260709 13:15:50.465133 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:50.466480 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:50.354596 14306 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:50.365823 14241 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.025s	sys 0.011s
I20260709 13:15:50.392524 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:50.480903 11518 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:50.484117 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.484479 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.484853 11512 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:15:50.485499 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 93059.8us
I20260709 13:15:50.486917 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.487739 12032 leader_election.cc:400] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7.
I20260709 13:15:50.393201 14359 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:50.494400 14391 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.463191 14009 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:50.395484 14371 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:50.496917 14227 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:50.497400 14227 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:50.497821 14080 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:50.498294 14227 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.500316 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.501154 14326 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:50.487771 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.356194 14372 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:50.506419 11436 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:50.486681 12034 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:50.363334 13844 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
W20260709 13:15:50.491307 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.395780 14351 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:50.415540 14376 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:50.509102 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 202896us
W20260709 13:15:50.487226 14241 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.029s	sys 0.011s
I20260709 13:15:50.416672 14018 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.088s
I20260709 13:15:50.517287 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.516634 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.483973 14352 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:15:50.523175 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.510540 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.353511 14303 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.491972 14357 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:15:50.463579 14223 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.419324 14284 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.419833 13953 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.601728 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.546166 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.601647 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:50.557507 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52568.9us
I20260709 13:15:50.628743 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:50.601912 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.777894 11517 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:15:50.778129 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.633502 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.774441 11514 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:50.636243 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.626366 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.615801 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.658608 11439 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:50.582049 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 95431.8us
I20260709 13:15:50.652201 11442 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:15:50.499356 14227 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:50.492599 14359 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.652115 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:50.598989 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:50.368628 14167 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:50.651346 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:50.779850 11516 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
W20260709 13:15:50.788430 11333 connection.cc:256] RPC call timeout handler was delayed by -1.0470084269998097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 13:15:50.789178 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.813661 11436 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:50.814158 11513 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:50.690120 14405 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:15:50.521860 14351 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.544870 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.788561 11369 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:50.817430 14332 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.822419 11512 pending_rounds.cc:85] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 2
I20260709 13:15:50.827970 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:50.813258 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 214349us
I20260709 13:15:50.597653 13661 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:50.837846 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1618) took 1228 ms. Trace:
I20260709 13:15:50.638756 14187 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.838742 11370 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:50.839080 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:50.835172 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.837958 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:50.646450 14364 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.195s
I20260709 13:15:50.841588 11517 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:50.812716 14359 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:50.631940 14213 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:50.544657 14391 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.496999 14371 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.525197 14171 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:50.634742 13913 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:50.856469 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2036) took 1052 ms. Trace:
I20260709 13:15:50.836078 14378 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:50.589722 14306 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.836822 11512 pending_rounds.cc:107] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 5.3
W20260709 13:15:50.866926 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 362228us
I20260709 13:15:50.844074 11369 rpcz_store.cc:276] 0709 13:15:49.609711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.615517 (+1005806us) service_pool.cc:224] Handling call
0709 13:15:50.837765 (+222248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:50.849841 11516 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:50.518857 14258 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.321s
I20260709 13:15:50.801394 14009 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:50.599225 14397 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.509455 14083 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.787914 14346 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:15:50.835860 14351 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:50.622882 14314 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.884284 11510 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:50.864194 13913 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:50.614204 14330 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.862784 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:50.904304 11436 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:50.868404 11513 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:50.849541 14317 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:50.866742 11512 raft_consensus.cc:2889] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:50.853835 14167 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.878124 14386 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.910029 14386 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.910845 14333 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:50.911275 14333 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:50.878577 14399 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:50.843492 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:50.859320 11438 rpcz_store.cc:276] 0709 13:15:49.803956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.599104 (+795148us) service_pool.cc:224] Handling call
0709 13:15:50.599939 (+   835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:50.624375 (+ 24436us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:50.624674 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 13:15:50.850502 (+225828us) log.cc:844] Serialized 14 byte log entry
0709 13:15:50.855857 (+  5355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:50.856150 (+   293us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:50.856198 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.856274 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.856402 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5122,"prepare.queue_time_us":3231,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":2444,"spinlock_wait_cycles":6655360,"thread_start_us":213655,"threads_started":1}
I20260709 13:15:50.853694 14353 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:50.692121 14362 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.865790 14364 consensus_queue.cc:1243] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:50.898198 14326 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:50.848875 14187 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.915904 14399 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:50.867655 14370 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:15:50.920390 14399 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:50.920727 14370 consensus_queue.cc:1243] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:50.909420 11442 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:15:50.882088 14409 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:50.921762 14223 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:50.921584 14009 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.922734 14284 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:50.842657 14332 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:50.852749 14080 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:50.921154 14083 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:50.908608 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:50.924600 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 81235.1us
I20260709 13:15:50.927717 11441 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:50.911653 14333 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:50.874089 14306 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.935776 11408 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:50.937705 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:50.916270 14317 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:50.943156 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:50.954042 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:50.930835 14401 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:51.012895 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:15:51.014756 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:51.021922 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.016333 12066 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:15:51.023978 14413 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:15:50.939724 14353 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:50.951380 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:50.934432 13661 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:50.915908 14167 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:50.943317 14083 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:51.017876 11516 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
W20260709 13:15:51.018710 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 83016.7us
I20260709 13:15:51.014668 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:50.884320 14397 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.024439 14396 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.951208 14171 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.952708 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:51.046057 14213 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.954233 14352 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.054625 14213 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:51.055488 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 115602us
I20260709 13:15:51.055930 14410 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.056391 14397 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.056464 14410 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.057044 11334 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:51.057760 14368 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:51.058478 14368 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:51.020648 14358 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.059571 14396 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.059888 13661 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.061178 14416 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.061728 14416 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.062521 14396 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:51.064136 14408 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:15:51.048985 14376 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:51.054284 14171 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.067235 14171 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:51.054247 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 102314us
I20260709 13:15:51.068583 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.070127 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:51.059005 14368 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:51.059346 14358 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.072399 14352 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:51.073508 14368 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.072346 14376 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:51.076141 14371 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.076402 14376 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:50.945909 14187 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:50.934346 14284 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 13:15:51.054934 14330 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.020887 14308 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:51.012037 14223 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:50.942451 14364 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:15:51.100327 11334 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:51.101150 14418 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.102036 14306 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.103509 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.115798 11445 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:51.044428 14401 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:51.058912 14410 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.050483 14386 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:51.121851 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 49534.8us
W20260709 13:15:51.123543 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 599499us
I20260709 13:15:51.124166 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.124962 11445 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:51.104249 14223 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:51.117560 14418 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.131701 14418 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.116724 14352 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.120467 14376 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:51.102869 14353 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.122156 14332 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:51.150611 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.128659 14223 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.109031 14308 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.157255 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.159235 14429 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 13:15:51.160055 14412 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:15:51.149642 14351 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.167562 14351 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.169912 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.170995 14429 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:51.171448 14429 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:51.173005 14420 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:51.173807 14430 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:51.174034 14420 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:51.174600 14420 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:51.174703 14430 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:51.175120 14430 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:51.088166 14326 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.175431 14420 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:51.175958 14430 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.177290 14420 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:51.173010 11333 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.179219 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:51.180032 14251 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:51.180984 14358 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.181159 14284 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:51.180027 14401 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.181620 14284 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.182058 14284 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:51.184702 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 84475us
I20260709 13:15:51.180733 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:51.192811 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.195080 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.149155 14416 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.177376 14430 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:51.204515 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.193634 11439 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
W20260709 13:15:51.207185 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 55308.1us
I20260709 13:15:51.124610 14317 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.228866 11439 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.232226 14326 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.232838 14326 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.235423 14326 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.205111 14368 raft_consensus.cc:2764] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:15:51.156443 14363 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:15:51.239538 14399 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.243898 14415 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:15:51.216593 14364 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:51.248531 11440 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:51.248732 14167 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:51.250177 14308 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.251286 14363 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:51.251928 14363 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:15:51.157048 14352 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.254603 14352 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.258498 11445 raft_consensus.cc:2410] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:51.262168 14352 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.264014 14352 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.264825 14407 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:51.265242 14407 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:51.266074 14407 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:51.274761 14372 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.280845 14372 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:51.281688 14365 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.282250 14365 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.284631 14365 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:51.299943 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 95511.4us
I20260709 13:15:51.280710 14407 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.274991 13913 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.349737 13913 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:51.351159 12064 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:51.288846 14284 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.374981 14399 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.247941 14359 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.494408 11438 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:51.448076 14223 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:51.462261 14399 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:51.500656 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.500283 14399 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:51.502276 14333 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.000s
I20260709 13:15:51.502843 14333 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:51.505302 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.506661 14284 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.513720 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:51.524928 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 330544us
I20260709 13:15:51.525399 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.526473 11436 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:51.530557 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:51.512518 14378 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.507593 14352 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.547952 11516 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:51.568406 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.550359 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:51.569607 11510 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:51.570221 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:51.570323 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:51.573791 11436 raft_consensus.cc:1217] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Deduplicated request from leader. Original: 2.5->[4.6-4.6]   Dedup: 4.6->[]
I20260709 13:15:51.574342 11436 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:15:51.530616 14223 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:51.571768 14401 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:51.575363 14378 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:51.566056 14399 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.578485 14359 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:51.593166 11334 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:51.590188 14401 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.597586 14430 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:51.601362 14333 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.593575 14439 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:15:51.606899 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.608853 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.604523 14401 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.610958 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.611095 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.611349 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.612383 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.612488 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.612648 14378 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.613730 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.617550 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.617491 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.620589 11365 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:51.618930 11361 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:51.620970 11519 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:51.620142 11368 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.628902 11519 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:51.631640 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.633003 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.617463 11362 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.647760 11437 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:51.648082 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.648826 11366 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:51.628841 11361 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:51.653347 11368 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:51.655252 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2046) took 1191 ms. Trace:
I20260709 13:15:51.657366 11441 rpcz_store.cc:276] 0709 13:15:50.464170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.485183 (+ 21013us) service_pool.cc:224] Handling call
0709 13:15:50.497506 (+ 12323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:50.932929 (+435423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.655006 (+722077us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54003151d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:51.655188 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987133696}
I20260709 13:15:51.617573 14241 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:51.648886 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.664530 11437 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:15:51.662729 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.662606 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:51.665506 11436 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
W20260709 13:15:51.668002 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:51.668768 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.668727 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.669912 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.666127 11519 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.671880 11519 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.674036 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.674712 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.670985 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.675796 11514 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:15:51.676362 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.676964 11441 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:51.674815 11516 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:51.681094 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:51.681321 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:51.682504 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.684762 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.686460 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.685331 11514 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:15:51.694067 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.694541 11368 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:51.695767 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:51.696147 12061 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:51.698098 11518 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:51.662935 14430 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.699752 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:51.700191 12065 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:51.700315 12069 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:51.701220 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:51.702749 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.692631 14365 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.703306 11366 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:51.703431 14365 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.705497 14365 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.695745 14358 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:51.707517 14358 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.709544 14358 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:51.711642 11361 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.712695 11361 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.712963 11518 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:51.714069 11518 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:51.696774 14258 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:15:51.701535 12069 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:51.696313 12068 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:51.667979 14241 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:51.726198 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.726935 11366 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:51.727811 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.728395 11366 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:15:51.729423 11519 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:51.730082 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1515) took 1120 ms. Trace:
I20260709 13:15:51.732681 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:51.730399 11519 rpcz_store.cc:276] 0709 13:15:50.609295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.818385 (+209090us) service_pool.cc:224] Handling call
0709 13:15:50.842771 (+ 24386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:51.620884 (+778113us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b54002551a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc975dc0 00007f35bca001c0 00007f35bca01231 00007f35bc9fdd8f 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c
0709 13:15:51.729828 (+108944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.730026 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149683584}
I20260709 13:15:51.735878 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:51.736366 11514 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.737113 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:51.743090 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.743633 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:51.744877 11442 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:51.746539 14442 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:15:51.747505 11367 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:51.747859 11367 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:51.748598 11367 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.749785 11367 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.751660 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:51.751977 14241 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:51.753257 14241 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.722924 14430 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:51.756958 11367 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:15:51.757949 14430 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:51.759099 11367 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:51.759713 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.769372 14429 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.770370 14429 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.772140 11514 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:51.775698 12061 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:51.776657 12065 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:51.765129 11367 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:51.766146 12069 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:51.738860 14018 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:15:51.767653 12068 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:51.769230 11442 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:15:51.783516 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.784893 11442 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:51.784061 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:51.790019 11262 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39460 (request call id 364) took 12902 ms (client timeout 15000 ms). Trace:
I20260709 13:15:51.792501 11266 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:51.794200 11266 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 2 to 3, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:51.612922 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:51.795011 11362 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:51.800714 14371 raft_consensus.cc:2764] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:15:51.804700 11511 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:15:51.829476 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.806334 14314 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.808486 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.831226 11363 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:51.833689 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:51.834002 13913 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:51.834586 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2034) took 2031 ms. Trace:
I20260709 13:15:51.835021 11440 rpcz_store.cc:276] 0709 13:15:49.803340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.460064 (+656724us) service_pool.cc:224] Handling call
0709 13:15:50.460379 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:51.262194 (+801815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.834394 (+572200us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54003142d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:51.834535 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812116864}
I20260709 13:15:51.809968 11370 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:51.835888 14303 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:51.836217 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:51.810159 14362 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:51.837790 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.838804 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.839565 11363 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:51.840348 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.842397 11361 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:51.843755 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.843968 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.845234 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.828709 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:51.846226 12064 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:51.820339 14401 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.851787 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.852483 11517 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:51.852730 11444 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:51.851393 11510 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:51.853837 14401 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.854373 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.854986 11511 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:51.856894 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:51.859542 14251 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:51.859887 14251 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.860014 11366 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:51.860215 14251 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:51.861092 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.830681 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.862011 11366 raft_consensus.cc:2410] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:15:51.862536 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.862824 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.813491 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:51.864701 14018 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:51.865331 14018 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:51.816510 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:51.868813 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.868458 14018 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.869475 11363 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:51.870040 11484 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:51.820370 14333 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.825801 14213 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:51.826893 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.828267 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.828435 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:51.834708 11443 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:51.858174 14439 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:51.809736 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:51.873605 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:51.812372 11364 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
W20260709 13:15:51.871922 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 240238us
I20260709 13:15:51.876108 11368 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:15:51.885529 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:51.870440 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.886068 11368 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:51.888010 11363 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.877655 11370 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
W20260709 13:15:51.879739 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51391us
I20260709 13:15:51.890619 12033 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:51.880601 11440 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:51.879261 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:51.896433 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:51.898741 11368 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:15:51.900334 11368 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:51.902428 12064 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:51.873824 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:51.907927 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.908712 11442 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:15:51.906531 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:51.911502 11370 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:51.911971 11444 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:51.912971 11364 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [5.4-9.6]
W20260709 13:15:51.913606 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 85459.8us
I20260709 13:15:51.915545 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:51.917903 11366 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:51.920528 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:51.920789 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:51.845533 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:51.903378 12068 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:51.877398 14430 consensus_queue.cc:1059] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:15:51.925482 11510 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:51.929623 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.930894 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:51.929553 11363 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
W20260709 13:15:51.930907 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.938423 14346 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:51.939648 14247 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:51.939924 14346 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:51.940441 14346 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:51.941622 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:51.942426 14247 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:51.942873 14247 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:51.942591 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.943204 14247 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:51.948115 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:51.941308 14346 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.949405 14346 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:51.959558 14247 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.961719 12034 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:51.963253 14418 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:51.963613 11361 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:51.964797 11516 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:51.964874 13661 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:51.965305 13661 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.932030 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:51.967706 11510 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:51.921680 11266 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:51.976046 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:51.976792 11515 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:51.977231 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.978051 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.904754 14314 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:51.904536 14358 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:51.980867 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:51.987596 11511 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:51.963171 11362 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:51.911788 14357 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:51.990674 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 75197.5us
I20260709 13:15:51.919884 14241 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:51.925110 14318 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:51.932068 14441 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:51.932376 14317 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:51.896191 14333 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.965641 13661 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.005673 11516 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:52.026042 11516 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:51.975675 14417 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:52.006372 14430 consensus_queue.cc:1243] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:51.920642 13844 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:15:52.007300 14346 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:52.008708 14247 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.010795 14399 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:52.017609 11511 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:15:52.003541 14441 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:52.021062 14223 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:51.907727 14167 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:15:51.985774 14314 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.994585 14357 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:52.093267 14357 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:52.093719 14399 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.094231 14399 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:51.998531 14241 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.095588 14241 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.095770 14441 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:52.063979 14424 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:15:51.790434 11262 rpcz_store.cc:269] 0709 13:15:38.887765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.888084 (+   319us) service_pool.cc:224] Handling call
0709 13:15:51.789805 (+12901721us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:46.485311 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:46.546492 (+ 61181us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:46.546532 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:15:46.546544 (+    12us) tablet.cc:662] Decoding operations
0709 13:15:46.547935 (+  1391us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:46.547989 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:46.548017 (+    28us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:15:46.549106 (+  1089us) tablet.cc:701] Row locks acquired
0709 13:15:46.549121 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:15:46.549263 (+   142us) write_op.cc:270] Start()
0709 13:15:46.549382 (+   119us) write_op.cc:276] Timestamp: P: 1783602946549216 usec, L: 0
0709 13:15:46.549398 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:15:46.602909 (+ 53511us) log.cc:844] Serialized 6586 byte log entry
0709 13:15:46.794884 (+191975us) op_driver.cc:464] REPLICATION: finished
0709 13:15:47.030231 (+235347us) write_op.cc:301] APPLY: starting
0709 13:15:47.034506 (+  4275us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:47.104549 (+ 70043us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:47.104666 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:47.208570 (+103904us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:47.208832 (+   262us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:15:47.208877 (+    45us) write_op.cc:312] APPLY: finished
0709 13:15:47.209217 (+   340us) log.cc:844] Serialized 113 byte log entry
0709 13:15:47.209388 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:47.209542 (+   154us) write_op.cc:454] Released schema lock
0709 13:15:47.209590 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:47.803957 (+594367us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54001897d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f9e28 00007f35c0434581 00007f35c0433d58 00007f35c04389f1 00007f35c04389a9 00007f35c0438939 00007f35c0438901 00007f35c0437b2d 000000000038f354 000000000038f199
0709 13:15:47.804167 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:47.804195 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":226401,"apply.run_cpu_time_us":8350,"apply.run_wall_time_us":185880,"num_ops":16,"prepare.queue_time_us":181528,"prepare.run_cpu_time_us":15460,"prepare.run_wall_time_us":1325502,"raft.queue_time_us":72023,"raft.run_cpu_time_us":15935,"raft.run_wall_time_us":87625,"replication_time_us":245337,"spinlock_wait_cycles":42123520,"thread_start_us":195274,"threads_started":4,"wal-append.queue_time_us":58399,"wal-append.run_cpu_time_us":8379,"wal-append.run_wall_time_us":1668016}]]}
I20260709 13:15:52.063570 14346 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.099786 14346 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.107916 11511 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:52.099457 14314 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.105360 14258 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.116744 14376 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:52.117487 14258 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.117920 14376 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:52.118295 14376 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:52.119055 14376 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:52.120216 14376 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.081497 11367 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:52.092322 11516 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.110347 14362 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:15:52.121045 11516 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:52.126251 14346 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.119606 11266 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 2 to 4, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.128028 14314 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.128347 11266 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 2 to 6, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.130151 14314 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.130759 11266 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 5 to 6, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:52.132609 11511 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:52.149950 11516 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:15:52.150604 11516 pending_rounds.cc:85] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 6
I20260709 13:15:52.150897 11516 pending_rounds.cc:107] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:15:52.151167 11516 raft_consensus.cc:2889] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:52.152503 11266 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 4, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.067292 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:52.156577 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 89387.8us
I20260709 13:15:52.160827 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:52.104823 14417 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:52.180688 14314 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:52.186522 14314 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:52.188429 14314 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.166060 13844 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:15:52.193217 14258 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:52.168876 14379 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:15:52.195263 14379 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:52.195578 14372 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:52.196035 14372 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:52.197001 14372 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.204463 14333 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:52.205384 14317 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:52.206061 14317 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:52.191697 14441 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.149933 14308 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:52.177268 11266 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 2, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.130647 13661 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:52.227008 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 298915us
I20260709 13:15:52.218020 14379 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:52.250206 11266 catalog_manager.cc:5697] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 4, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.198340 14372 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.233760 14358 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:52.229377 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 294875us
I20260709 13:15:52.235733 13913 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:52.179924 14417 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.254184 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:52.262362 11266 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 4, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:52.266513 12063 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:15:52.266948 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:52.256036 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.225162 14371 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:52.236708 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 305861us
W20260709 13:15:52.272413 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.276080 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.273113 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:52.268035 11445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.010s	sys 0.030s
W20260709 13:15:52.277223 11445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.010s	sys 0.031s
I20260709 13:15:52.225072 14251 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:52.284001 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 322280us
I20260709 13:15:52.291641 12064 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:15:52.293431 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 63617.5us
I20260709 13:15:52.295423 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:52.292616 11266 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 2 to 4, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 4 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:15:52.341192 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 64376us
I20260709 13:15:52.290642 12034 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:52.351383 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:52.329751 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.382606 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.384326 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 92705us
I20260709 13:15:52.385293 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:52.386327 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:15:52.391498 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.404632 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.383942 11266 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:52.294834 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:52.425427 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:52.428599 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 140093us
I20260709 13:15:52.294612 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:52.432664 11510 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:52.281121 14371 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:52.438660 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 149354us
I20260709 13:15:52.446313 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:52.447880 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.448343 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.452734 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:52.295676 14357 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.416044 14258 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:15:52.458849 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.383826 14241 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:52.439836 11362 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.430840 11361 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:52.465956 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:52.420614 14439 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:15:52.431244 14457 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:52.296939 14379 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.451136 14465 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:15:52.493978 14456 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:15:52.494585 14465 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:52.450856 14407 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.496934 14407 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:52.450071 14399 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:52.450018 14346 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.458796 14463 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.456521 14317 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.498536 11362 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:52.464643 14466 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:52.483489 12062 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:52.457849 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:52.533563 11512 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 13:15:52.474224 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 178881us
I20260709 13:15:52.432485 14358 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.536386 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:52.476665 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:52.471908 14213 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:52.551528 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.552277 12069 raft_consensus.cc:2368] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6.
I20260709 13:15:52.480082 14223 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:52.530689 13661 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:52.554600 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.532357 14399 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:52.556190 14481 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:52.558781 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:52.541756 14247 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.480006 14472 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:52.563813 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 177387us
I20260709 13:15:52.479261 13844 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:52.549599 14318 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:52.549839 14379 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.566546 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:52.567099 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:52.564198 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 111534us
I20260709 13:15:52.565747 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:52.495090 14465 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.568579 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:52.569896 12069 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:52.551626 14473 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:52.553932 14223 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:52.561916 14213 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:52.577064 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:52.534880 14463 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.577881 11439 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:52.590236 14461 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:52.590413 14223 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:52.591686 14317 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:15:52.595896 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.596921 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.598914 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.599792 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.694964 12061 raft_consensus.cc:2410] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:15:52.600576 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.572177 14251 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.625198 14371 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:52.630626 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:52.633991 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:52.635061 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.718170 12066 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:52.642119 13913 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:52.719316 13913 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:52.642885 14308 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:52.563750 14417 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:52.649226 14468 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:15:52.569134 14399 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:52.656389 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 79455.3us
W20260709 13:15:52.666282 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 92684.1us
W20260709 13:15:52.667052 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 99257.8us
I20260709 13:15:52.733282 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:52.572284 14482 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.573982 14459 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:52.714968 12067 raft_consensus.cc:2368] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12.
I20260709 13:15:52.581065 14481 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:52.579133 14407 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:52.716986 12069 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:52.579771 13844 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:52.729106 12060 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:52.734243 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:52.674786 14477 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:52.735244 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:52.741460 11515 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:52.738185 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:52.743275 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:52.722759 13661 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.656365 14473 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:52.727139 14483 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:52.748147 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:52.730760 14308 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:52.747740 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:52.755853 11439 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:52.760133 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:52.760667 11439 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:52.764299 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:52.765918 11334 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:52.767194 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:52.738090 14241 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.768002 11442 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:52.772576 11367 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:15:52.774959 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.739843 14458 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:52.751832 14346 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:52.742118 14459 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:52.817549 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:53.001883 11442 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:52.764020 14357 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:52.776193 13661 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.005769 13661 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.908116 14463 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:53.007153 12063 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:52.911898 14018 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:52.913102 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:52.913280 14473 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.913488 14318 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:52.777872 14466 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
I20260709 13:15:52.773178 14382 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:53.010103 14382 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.012125 14382 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:52.914314 14465 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:52.914602 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.019769 11442 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:52.777365 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:52.920413 14401 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:52.782727 14213 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.784637 14480 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 13:15:52.830235 13844 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.927639 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:52.762436 14479 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.036707 14479 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.038877 14479 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:52.930241 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:53.039588 12060 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:52.930917 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:53.040740 12068 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:52.932049 12064 raft_consensus.cc:2410] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:15:52.780673 14457 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:52.932363 14284 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:53.041788 14284 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:52.932803 14441 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.708s	user 0.005s	sys 0.005s
I20260709 13:15:52.780226 14370 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:52.954906 14353 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:15:53.044509 14353 consensus_queue.cc:1243] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:53.044976 14353 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.045429 14353 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.933571 12034 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:52.921937 13913 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:52.774411 14258 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:15:52.936517 14399 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:52.962530 14167 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.773464 14407 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:52.773566 14484 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.774610 14371 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:15:52.781783 14372 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:15:52.937551 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:52.763170 14251 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:52.971426 11439 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
W20260709 13:15:52.978096 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.941900 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 306811us
I20260709 13:15:53.047350 14353 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:52.949682 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.941212 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 155803us
I20260709 13:15:53.008994 11440 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:53.034395 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:53.065820 11441 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:15:53.040993 14446 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 1.164s	user 0.015s	sys 0.016s
W20260709 13:15:53.047751 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 114272us
I20260709 13:15:52.937508 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:53.070374 11436 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:52.961938 14439 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.974399 14317 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:53.053092 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:53.000648 14241 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:53.047936 14346 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.053656 14477 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:53.057796 14371 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:53.057106 14167 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.057960 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:52.991820 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 261388us
I20260709 13:15:53.044986 14223 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.066817 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:53.095968 11439 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:53.054263 14352 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:53.068209 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.069809 12034 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:53.055878 14407 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:53.078717 14353 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:15:53.091513 14314 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.002s	sys 0.018s
I20260709 13:15:53.138420 12067 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:15:53.145483 14372 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.146044 14372 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.150249 14284 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.159291 14284 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:53.164110 14441 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.164732 14441 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.173943 14372 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:53.175436 14493 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.176126 14493 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.125839 14371 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:53.180374 14371 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.183729 14477 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.082662 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:53.061761 13913 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:53.093452 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:53.106913 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:53.107530 14333 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:53.108055 11438 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:53.195448 11436 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
W20260709 13:15:53.193909 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 111311us
I20260709 13:15:53.196693 11443 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 13:15:53.197018 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.197118 12033 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:53.197789 11436 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:53.179138 14493 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:53.200807 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 107427us
I20260709 13:15:53.103886 14352 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.209816 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:53.210582 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 433268us
I20260709 13:15:53.107945 14481 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:53.218055 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 111234us
I20260709 13:15:53.218658 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.126726 14407 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:53.216305 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:53.219784 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:53.126423 14353 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:53.059762 14484 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.209949 14359 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:53.289908 11511 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:53.244980 12069 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:53.191468 14483 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:53.291738 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 72039.2us
I20260709 13:15:53.080505 14439 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.246850 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.093241 14241 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.254118 12060 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:53.251936 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:53.259001 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:53.244060 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.304509 11370 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:53.305054 11367 raft_consensus.cc:2368] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8.
I20260709 13:15:53.244060 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:53.305400 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.253127 11369 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:53.306115 11366 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:53.274219 12067 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:15:53.275576 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 78542.2us
I20260709 13:15:53.279790 11445 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:15:53.284021 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.311685 11516 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:53.085431 14317 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:53.284920 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.210978 14401 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:53.315078 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.216907 14352 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:53.292156 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:53.283742 14489 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:53.217964 14333 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:53.296751 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 231706us
I20260709 13:15:53.331700 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:53.331820 14333 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.332711 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:53.276008 14446 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:53.297453 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.328447 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:53.297101 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.246608 14372 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.337944 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:53.209167 14458 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:53.339958 11515 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:53.340293 14458 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:15:53.315270 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.324761 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.221487 14418 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:53.254593 14370 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:53.328135 12064 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:15:53.273625 14399 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.315896 11519 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:53.342188 14258 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:53.342473 11445 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:53.337213 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.343185 14258 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:53.343822 14258 consensus_queue.cc:1243] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:15:53.319948 14247 consensus_meta.cc:287] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.000s
I20260709 13:15:53.343254 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.297437 14358 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:53.344772 14358 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.344901 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:53.337456 11517 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:15:53.340922 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:53.345311 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.346400 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.347198 11515 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:15:53.337549 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:53.342145 14399 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.349880 14399 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:53.341965 14370 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.343339 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.352218 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.344338 14247 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:53.345177 14358 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.355541 11517 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:15:53.358639 14258 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.359216 14258 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:53.361055 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 50431.6us
I20260709 13:15:53.361234 11366 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:53.361825 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:53.363480 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:53.297569 14379 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:53.297257 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:53.365635 11512 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.317255 14460 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:53.373435 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:53.374663 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:53.384236 11408 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:53.328191 14018 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.413298 12063 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:53.338930 14359 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:53.416548 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 54814.8us
W20260709 13:15:53.416734 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 71923.4us
I20260709 13:15:53.417264 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.427474 11366 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:15:53.427906 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1916) took 1914 ms. Trace:
W20260709 13:15:53.431342 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 98705.3us
I20260709 13:15:53.432601 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:53.428297 11513 rpcz_store.cc:276] 0709 13:15:51.513953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.514061 (+   108us) service_pool.cc:224] Handling call
0709 13:15:51.514391 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:51.524823 (+ 10432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:51.524854 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:51.572548 (+ 47694us) log.cc:844] Serialized 26 byte log entry
0709 13:15:51.635113 (+ 62565us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:51.640059 (+  4946us) log.cc:844] Serialized 14 byte log entry
0709 13:15:51.640474 (+   415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.640553 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:52.802053 (+1161500us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:53.418260 (+616207us) raft_consensus.cc:1685] finished
0709 13:15:53.427703 (+  9443us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:53.427845 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4483,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":10237,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":38877,"spinlock_wait_cycles":342608768,"thread_start_us":51980,"threads_started":1,"wal-append.queue_time_us":297483}
I20260709 13:15:53.376485 14461 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.367558 14418 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:53.339993 14372 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:53.446601 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 72018.7us
W20260709 13:15:53.429163 14471 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:53.408340 14441 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:53.459179 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.465090 11366 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:53.375984 14459 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:53.428153 14382 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:53.424018 14489 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:53.443277 14490 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
I20260709 13:15:53.443876 14472 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:53.428462 14460 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:53.434808 14314 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:53.447504 14465 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.484275 11362 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:53.387681 14379 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.478950 11334 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:53.441630 14505 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.464399 14371 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.416354 14513 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:15:53.377064 14353 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.554126 11436 raft_consensus.cc:3055] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:53.438989 14018 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.490589 14472 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.484258 14511 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:53.579941 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 101135us
I20260709 13:15:53.562103 11436 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:53.582060 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.564415 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.583133 11363 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:15:53.564453 12066 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:53.566346 14317 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:53.548779 14508 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.567374 14371 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.604611 11512 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
W20260709 13:15:53.571826 14494 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.242s	user 0.000s	sys 0.000s
I20260709 13:15:53.555716 14465 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.572023 14223 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:53.572471 14480 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:53.576114 14530 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:53.575729 14018 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:53.557449 14505 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.576653 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:53.566670 14516 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:15:53.578792 14482 consensus_meta.cc:287] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.836s	user 0.008s	sys 0.000s
I20260709 13:15:53.581110 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.590176 14441 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:53.541814 14314 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.593451 14511 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.474488 14483 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.599308 14460 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.600595 14251 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:53.602519 14258 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:53.603111 14526 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:53.539618 14382 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:53.605571 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:53.606631 11441 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.001s
I20260709 13:15:53.551605 14167 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:53.458700 14461 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.624089 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.604255 11436 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.574795 14379 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:53.593977 14472 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.593127 14370 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.715969 14370 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.716809 14461 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:53.673183 14251 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:53.718050 14370 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:53.639247 11516 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:53.630445 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.719347 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.720031 11511 pending_rounds.cc:85] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 5
I20260709 13:15:53.720351 11511 pending_rounds.cc:107] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:15:53.720569 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.639781 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:15:53.721766 14308 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.945s	user 0.010s	sys 0.000s
I20260709 13:15:53.642544 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1527) took 1712 ms. Trace:
I20260709 13:15:53.439961 14481 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.560653 14359 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.722977 11361 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:53.645313 11368 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:53.723708 14284 consensus_meta.cc:287] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.002s
I20260709 13:15:53.724049 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:53.724925 14498 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:15:53.725109 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.725776 14358 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.631388 14418 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.728003 14358 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:53.592625 14507 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.630610 14480 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:53.728718 14505 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:53.729185 14480 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.672708 11441 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:53.693967 11436 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:53.730248 14465 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:53.720510 14487 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.731372 14487 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.695819 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.696506 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:53.722338 11514 rpcz_store.cc:276] 0709 13:15:51.930048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.932018 (+  1970us) service_pool.cc:224] Handling call
0709 13:15:51.932312 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:52.226694 (+294382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:52.226741 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:52.864697 (+637956us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b2c000fe208. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:52.938402 (+ 73705us) log.cc:844] Serialized 26 byte log entry
0709 13:15:53.284713 (+346311us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:53.284750 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.284817 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:53.642220 (+357403us) raft_consensus.cc:1685] finished
0709 13:15:53.642241 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:53.642480 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88237,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"spinlock_wait_cycles":88517504,"thread_start_us":639277,"threads_started":2,"wal-append.queue_time_us":638576}
I20260709 13:15:53.720577 11511 raft_consensus.cc:2889] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:53.736253 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.738036 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.664281 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:53.693113 14526 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:53.709437 14524 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:15:53.634449 14489 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.726143 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.625159 14353 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:53.698716 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:53.660565 14479 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.660257 14314 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:53.659947 14482 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:53.752595 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.774003 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:53.628747 14508 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.755637 14531 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:53.714710 14382 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.662606 14511 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.759106 11436 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:53.743681 14167 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:53.722182 14528 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:53.794905 14528 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.662897 14365 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:53.795876 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 97244.8us
I20260709 13:15:53.796103 14365 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.797034 14520 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:53.798312 14365 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:53.763597 11368 raft_consensus.cc:2410] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:15:53.766762 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:53.801059 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.692991 14527 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.801743 14527 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:53.802932 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.730306 14480 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:53.803803 11436 raft_consensus.cc:2410] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:15:53.804908 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.806578 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:53.807638 11512 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:53.808238 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:53.776397 14317 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:53.770670 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:53.785046 11366 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:53.821570 11512 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:53.822469 14465 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:53.815179 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 546) took 1065 ms. Trace:
I20260709 13:15:53.823797 11437 rpcz_store.cc:276] 0709 13:15:52.749726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.749839 (+   113us) service_pool.cc:224] Handling call
0709 13:15:52.767579 (+ 17740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:53.000110 (+232531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:53.000138 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:53.022960 (+ 22822us) log.cc:844] Serialized 26 byte log entry
0709 13:15:53.262582 (+239622us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:53.262625 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.262698 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:53.814974 (+552276us) raft_consensus.cc:1685] finished
0709 13:15:53.814994 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:53.815119 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":226860,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"thread_start_us":423693,"threads_started":2,"wal-append.queue_time_us":678369}
I20260709 13:15:53.824589 14526 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.722180 14308 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.828322 11513 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:53.829408 14308 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:53.827733 14472 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.835512 11366 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:53.761512 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:53.790266 11440 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:53.718008 14251 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.637939 14223 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:53.798552 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:53.777245 14461 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.696695 14495 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.159s
I20260709 13:15:53.797101 14528 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:53.777683 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 113487us
I20260709 13:15:53.728737 14507 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.164381 14376 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:53.788369 14370 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:53.904062 14482 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:54.165781 11445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.019s	sys 0.008s
W20260709 13:15:53.852893 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.065557 12069 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:53.812160 14535 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:54.167227 14535 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.166862 14223 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.169520 14535 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:54.070215 14540 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 13:15:54.170477 11445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.019s	sys 0.008s
I20260709 13:15:54.073745 11369 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:53.832895 14417 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.333s
I20260709 13:15:53.896795 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:54.111196 14527 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:53.996084 14308 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.164618 14483 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:53.832664 14167 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.036166 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:54.173760 14483 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.163863 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.175034 14167 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:54.017760 14407 consensus_meta.cc:287] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.741s	user 0.009s	sys 0.001s
I20260709 13:15:54.173707 14308 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.176055 14461 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.177574 14483 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:53.920393 14511 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.938827 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:54.178894 11363 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:54.175241 11362 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:54.179576 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.173259 11361 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.180579 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:15:54.165390 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 403967us
I20260709 13:15:54.175809 14407 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:53.898416 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 58299.3us
I20260709 13:15:54.176237 11518 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.183656 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:54.109493 14399 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:53.777015 14479 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:53.904222 14465 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.267s
I20260709 13:15:54.178194 14308 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.151808 14457 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:54.165159 14370 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:54.203285 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 404791us
I20260709 13:15:54.207650 11483 leader_election.cc:400] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6.
I20260709 13:15:54.208006 11483 leader_election.cc:403] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:54.191373 14524 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:54.201807 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.182927 14511 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.194634 14538 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.232730 14538 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.234750 14538 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.235563 14371 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:54.215220 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.216025 14461 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.216532 14372 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:54.217068 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.217635 14515 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6.
I20260709 13:15:54.219614 14439 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:54.223048 14352 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.886s	user 0.002s	sys 0.004s
I20260709 13:15:54.201524 14418 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:54.226729 14460 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:54.227761 14487 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:54.230947 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:54.232098 14508 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.195192 14482 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.239118 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.239123 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.241803 11361 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
W20260709 13:15:54.215946 14542 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:15:54.262148 14439 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.245316 11518 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:54.286175 14439 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.202075 14399 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.248916 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.250990 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.288159 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.288666 14399 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:54.289239 14399 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:54.289825 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.290658 14527 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:54.291010 14353 consensus_meta.cc:287] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.000s
I20260709 13:15:54.291261 14527 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:54.253394 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.254827 14167 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:54.206401 14457 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:54.293592 14479 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:54.173019 14507 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.291673 14353 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:54.290999 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:54.296000 14353 raft_consensus.cc:2764] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:15:54.297418 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.271222 11510 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:54.264539 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:54.257941 14359 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.007s	sys 0.024s
I20260709 13:15:54.293195 14457 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.254310 14472 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.238499 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.305428 11519 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:15:54.263130 14352 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:54.480319 11484 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.845s (SENT) [suppressed 11 similar messages]
I20260709 13:15:54.298553 14353 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:54.270622 14371 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:54.481968 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 233124us
W20260709 13:15:54.304838 14359 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.008s	sys 0.025s
I20260709 13:15:54.482770 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.210359 14534 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:15:54.481835 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 184488us
I20260709 13:15:54.299741 14522 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:54.311794 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 73371.4us
I20260709 13:15:54.467129 14510 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.335127 14258 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:54.345361 14457 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.434890 14441 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:54.472465 14553 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:15:54.397347 14508 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:15:54.486408 11518 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:54.485334 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 556) took 1191 ms. Trace:
I20260709 13:15:54.485719 14483 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:54.320698 14382 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:15:54.465606 14546 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.010s	sys 0.017s
I20260709 13:15:54.493120 11483 leader_election.cc:400] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12.
I20260709 13:15:54.512113 11483 leader_election.cc:403] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:54.346110 14472 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.345165 14376 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.484624 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:54.258932 14535 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.238s
I20260709 13:15:54.509050 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.304607 14418 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:54.513427 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.496381 14407 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.519264 11517 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:15:54.511857 14546 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.010s	sys 0.018s
I20260709 13:15:54.521916 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.521994 11510 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:54.520686 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:54.508970 14382 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.528133 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:54.533138 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.506446 11445 rpcz_store.cc:276] 0709 13:15:53.294084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.354679 (+ 60595us) service_pool.cc:224] Handling call
0709 13:15:53.364306 (+  9627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.367691 (+  3385us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:53.416509 (+ 48818us) log.cc:844] Serialized 14 byte log entry
0709 13:15:54.484975 (+1068466us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:54.485032 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:54.485076 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.485145 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:54.485271 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74202,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":3958,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":96,"thread_start_us":1068111,"threads_started":2,"wal-append.queue_time_us":881896}
I20260709 13:15:54.536084 11362 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:54.498242 14353 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:54.530519 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.489814 14371 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:54.537184 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56911.3us
I20260709 13:15:54.537338 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.541714 11362 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:54.542619 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.543583 14372 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:54.543906 11362 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:54.543979 14372 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.544414 14372 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.521384 14524 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:54.544840 14524 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:54.545698 14524 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.546830 14524 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:54.547732 11444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2320) took 1632 ms (client timeout 1845 ms). Trace:
W20260709 13:15:54.548101 11444 rpcz_store.cc:269] 0709 13:15:52.914881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.927358 (+ 12477us) service_pool.cc:224] Handling call
0709 13:15:54.547529 (+1620171us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000839d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:54.547648 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2488527232}
I20260709 13:15:54.523597 14399 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:15:54.550045 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.521873 14370 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.501860 14522 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.518530 14258 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:54.552055 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.503278 14510 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.581040 14510 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:54.584115 14482 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.584645 14458 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.585754 14458 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:54.536827 14481 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:54.537539 14382 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:54.589278 14308 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:54.589684 14308 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:54.590437 14308 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:54.557677 14547 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:15:54.537825 14495 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 13:15:54.558845 14556 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:15:54.559540 14557 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:15:54.559810 14522 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:54.560309 14554 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:15:54.560628 14258 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.561452 11361 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:54.561923 11370 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:54.563299 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.603936 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 797) took 1288 ms. Trace:
W20260709 13:15:54.563824 14541 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.738s	user 0.000s	sys 0.003s
I20260709 13:15:54.523775 14376 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:54.568071 14545 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:15:54.568228 14531 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.561482 14439 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:54.523164 14534 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12.
I20260709 13:15:54.573709 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2319) took 1659 ms. Trace:
I20260709 13:15:54.577927 14353 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.519655 14441 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.591377 14308 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.604440 12069 rpcz_store.cc:276] 0709 13:15:53.315531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.316123 (+   592us) service_pool.cc:224] Handling call
0709 13:15:53.316515 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:54.166959 (+850444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.603660 (+436701us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b540052b7d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:54.603875 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1721844096}
I20260709 13:15:54.522310 14472 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.605980 11510 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:15:54.564538 14550 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:15:54.606869 11517 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:54.610881 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47668.2us
I20260709 13:15:54.585882 14371 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.618547 11440 rpcz_store.cc:276] 0709 13:15:52.914594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.065184 (+150590us) service_pool.cc:224] Handling call
0709 13:15:53.065488 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:54.164971 (+1099483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.573454 (+408483us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54001e9e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:54.573647 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666335104}
I20260709 13:15:54.616751 14352 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:54.620193 14314 consensus_meta.cc:287] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.831s	user 0.008s	sys 0.003s
I20260709 13:15:54.611274 14399 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:54.618259 14538 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:54.652350 14538 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.652817 14538 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:54.616539 14531 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.630151 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:54.647459 14314 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:54.663138 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.664947 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1645) took 1399 ms. Trace:
I20260709 13:15:54.624600 14353 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.664529 14534 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:54.667459 11364 rpcz_store.cc:276] 0709 13:15:53.265434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.266178 (+   744us) service_pool.cc:224] Handling call
0709 13:15:53.594617 (+328439us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:15:53.594830 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.595017 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:54.052967 (+457950us) log.cc:844] Serialized 14 byte log entry
0709 13:15:54.664626 (+611659us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:54.664676 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:54.664730 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.664790 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:54.664895 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":716510,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":61903,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4435,"spinlock_wait_cycles":14058240,"thread_start_us":1021019,"threads_started":3,"wal-append.queue_time_us":571366}
I20260709 13:15:54.685570 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:54.680569 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 151265us
I20260709 13:15:54.681519 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.687891 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.688146 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.681949 11362 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:54.689580 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:54.690075 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:54.690243 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.690510 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.690928 14495 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:54.691046 11362 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:54.692015 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.692209 11409 leader_election.cc:400] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8.
I20260709 13:15:54.691465 14495 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.684401 14371 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.692550 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:54.693562 11362 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:54.757629 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:54.699016 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.699858 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 674) took 1251 ms. Trace:
I20260709 13:15:54.759809 11513 rpcz_store.cc:276] 0709 13:15:53.448384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.448591 (+   207us) service_pool.cc:224] Handling call
0709 13:15:53.448899 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.828682 (+379783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.699656 (+870974us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54000e9f54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:54.699805 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":872171264}
I20260709 13:15:54.700311 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 686) took 1219 ms. Trace:
I20260709 13:15:54.761229 11368 rpcz_store.cc:276] 0709 13:15:53.480595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.801135 (+320540us) service_pool.cc:224] Handling call
0709 13:15:53.801363 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.838005 (+ 36642us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:53.838041 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:53.838088 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:53.838126 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.700111 (+861985us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b54002db754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:54.700152 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:54.700255 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83624704}
I20260709 13:15:54.709429 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.763666 11445 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:54.714644 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.715582 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.766417 11514 raft_consensus.cc:2410] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:15:54.718847 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.721689 12061 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:54.721998 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.726298 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.727603 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:54.727941 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.770406 11439 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.768913 11511 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:54.729511 11361 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:15:54.730325 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.734437 11408 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:54.692742 11409 leader_election.cc:403] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:54.738076 11333 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.724314 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.728443 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.720058 12065 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:15:54.773872 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.774098 11443 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:54.727375 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.774945 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:54.775182 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.724804 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.714365 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.715306 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.722764 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1931) took 1168 ms. Trace:
I20260709 13:15:54.736016 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:54.768704 12060 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:54.744527 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.722463 12063 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:54.715883 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:54.780652 11516 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:54.769438 12067 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 3.
I20260709 13:15:54.781769 11512 raft_consensus.cc:2410] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:15:54.782138 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.715694 14382 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.782902 11368 raft_consensus.cc:2410] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:15:54.715857 14308 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.696660 14223 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:15:54.737313 14543 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.268s	user 0.001s	sys 0.001s
I20260709 13:15:54.784140 11368 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:54.701261 14372 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.785307 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1647) took 1499 ms. Trace:
I20260709 13:15:54.694262 14376 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:54.738722 14548 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:15:54.742142 12033 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:54.714155 11519 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:15:54.744577 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.722750 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:54.778102 11515 rpcz_store.cc:276] 0709 13:15:53.554395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.611341 (+ 56946us) service_pool.cc:224] Handling call
0709 13:15:53.634531 (+ 23190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.796485 (+161954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:53.796515 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:53.796554 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:53.796583 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.722567 (+925984us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540031f954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:54.722610 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:54.722714 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372036352}
W20260709 13:15:54.788316 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.707723 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.723030 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.719097 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.714228 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2188) took 1255 ms. Trace:
I20260709 13:15:54.773464 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:54.788813 11443 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:54.781127 11366 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [5.4-9.6]
I20260709 13:15:54.722539 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 685) took 1248 ms. Trace:
I20260709 13:15:54.777590 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:54.818830 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.716215 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:54.820107 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 84166.9us
I20260709 13:15:54.758177 11362 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.744174 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:54.792721 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:54.789247 12068 raft_consensus.cc:2368] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6.
I20260709 13:15:54.822433 11361 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:15:54.823856 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.824122 11439 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
W20260709 13:15:54.819645 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 77580.5us
I20260709 13:15:54.825335 11511 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:54.826423 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.821043 11369 rpcz_store.cc:276] 0709 13:15:53.459121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.459480 (+   359us) service_pool.cc:224] Handling call
0709 13:15:54.073539 (+614059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:54.173942 (+100403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.714055 (+540113us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54000c6f50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:54.714176 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364463488}
I20260709 13:15:54.827075 11514 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:54.789034 14317 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:54.697125 14371 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.830856 13913 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:54.831445 13913 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:54.832042 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:54.832634 12069 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.723996 14484 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.832455 13913 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.833361 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.743414 14358 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:15:54.835803 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.789340 14223 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:54.838337 14505 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.838210 14223 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.788089 14510 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:54.839706 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.839840 14507 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:54.840834 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.841424 12061 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:54.842247 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.842803 12061 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:54.843663 14317 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:54.844439 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.844827 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:54.844642 14317 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.845695 11515 raft_consensus.cc:2410] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:54.846766 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:54.847283 12062 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:54.847676 14247 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:54.848708 12065 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:54.697415 14441 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.850392 14223 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:54.851882 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 74374.4us
I20260709 13:15:54.839730 14510 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.853125 14510 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.853371 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.693722 14418 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.693799 14472 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.695681 14353 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.852483 14247 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:54.786028 14372 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:54.817061 14284 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:54.698652 14314 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:54.701696 14399 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.698719 14572 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.903748 14572 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.905926 14572 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.852138 14568 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.906579 14568 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.848489 14483 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:54.778908 11517 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:54.790345 11368 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:15:54.848737 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 104638us
I20260709 13:15:54.907577 14483 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:54.907886 14223 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:54.908070 14483 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:54.908459 14223 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.799098 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.910162 11436 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:54.910610 14223 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:54.818687 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45285.5us
I20260709 13:15:54.824958 11512 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:15:54.884795 14418 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.832845 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.860203 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:54.911294 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:54.785635 11365 rpcz_store.cc:276] 0709 13:15:53.286275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.286380 (+   105us) service_pool.cc:224] Handling call
0709 13:15:53.286693 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:53.480859 (+194166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:53.480889 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:53.481324 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:15:54.090094 (+608770us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:54.116383 (+ 26289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.602477 (+486094us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b5400249854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:54.602529 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:54.736802 (+134273us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:15:54.785105 (+ 48303us) raft_consensus.cc:1685] finished
0709 13:15:54.785126 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:54.785255 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20092,"raft.queue_time_us":32758,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":18738,"spinlock_wait_cycles":130004224,"thread_start_us":643541,"threads_started":2,"wal-append.queue_time_us":918449}
I20260709 13:15:54.863870 12069 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:54.860852 11513 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:15:54.913777 11513 pending_rounds.cc:85] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 5
I20260709 13:15:54.913986 14489 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:54.914695 14481 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:54.913671 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:54.866885 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 174793us
I20260709 13:15:54.914016 11441 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:54.825615 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.857278 11363 rpcz_store.cc:276] 0709 13:15:53.474386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.630187 (+155801us) service_pool.cc:224] Handling call
0709 13:15:54.722379 (+1092192us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54001e6ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:54.722482 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057764224}
I20260709 13:15:54.918437 11443 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:54.870123 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:54.919090 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 49053.7us
I20260709 13:15:54.919713 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:54.920070 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:54.829270 12033 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:54.694793 14495 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.716502 14241 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.865412 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:54.924389 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59061.8us
I20260709 13:15:54.884765 11362 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:54.923919 14352 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:54.923955 14439 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:15:54.885514 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 64668us
I20260709 13:15:54.886327 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.871975 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.889113 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:54.926932 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.772506 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:54.733778 14251 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.890434 11366 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:15:54.716123 14531 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:54.838865 14505 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:54.893862 14576 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:15:54.785576 11370 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:54.911859 11512 pending_rounds.cc:85] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 7
I20260709 13:15:54.913415 14418 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.914072 11513 pending_rounds.cc:107] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:15:54.952039 11513 raft_consensus.cc:2889] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:54.916133 14481 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:15:54.865517 11516 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:54.723637 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:54.920650 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.953758 11512 pending_rounds.cc:107] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:15:54.892084 14284 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:54.703660 14538 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:54.867971 14477 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8.
I20260709 13:15:54.922425 11334 leader_election.cc:400] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6.
I20260709 13:15:54.916949 11408 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:54.921996 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:54.921417 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.925027 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:54.925531 14352 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.954095 11512 raft_consensus.cc:2889] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:54.967769 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 50901us
I20260709 13:15:54.961087 11445 raft_consensus.cc:2410] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:54.926011 14358 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:54.925294 14572 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:54.950666 14241 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:54.887758 14524 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:15:54.966187 11334 leader_election.cc:403] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:15:54.966687 11444 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.919353 14489 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:54.925822 14439 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:54.970526 11443 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:15:54.950714 14314 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:54.971360 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.954384 14353 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:54.966223 14382 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:54.927641 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55766.4us
I20260709 13:15:54.970914 14352 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.966825 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:54.966614 11442 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:54.953022 14570 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.927380 11515 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:54.963490 14284 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:54.992756 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70458.2us
I20260709 13:15:54.996302 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.999058 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:54.925863 14482 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:15:55.058327 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.003454 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.004861 11436 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:54.978869 14241 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:54.973120 14559 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:55.047624 14511 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:55.048173 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.952652 14223 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:15:54.970332 14371 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:54.968730 14418 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:55.052395 11444 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:55.052506 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:55.056471 14479 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:55.086522 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.086792 11335 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:55.053421 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 128374us
I20260709 13:15:55.053934 14477 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:54.961594 14308 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:15:54.978659 14572 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.058271 11334 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:55.100306 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:55.102193 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:55.060776 11440 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 4.
I20260709 13:15:55.019145 11333 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:54.991305 14534 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:55.062237 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.002955 14372 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6.
I20260709 13:15:55.004518 14314 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:54.921343 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 92140.9us
I20260709 13:15:55.002851 14439 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:55.017932 14353 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:55.115299 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:55.116164 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 191204us
I20260709 13:15:55.117524 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:55.073482 11442 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:15:55.088631 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:54.978132 14563 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:15:55.091298 11335 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:55.092115 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:55.096339 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.066699 14241 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.101527 14371 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:55.113063 12033 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:55.135000 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 43772.8us
I20260709 13:15:55.114173 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.066944 14559 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:55.058395 14570 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:55.144838 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 183035us
I20260709 13:15:55.114362 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.124621 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:55.150058 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:55.085857 14418 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:55.160414 11440 raft_consensus.cc:2410] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:15:55.095224 14376 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:55.104063 14572 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:55.090464 14477 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:55.111901 14522 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:55.110996 14223 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.135838 14353 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:55.116257 14314 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.085527 14511 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.100486 14481 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:15:55.141571 14441 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:55.160507 14466 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.141113 14482 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:55.141305 14371 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.141922 14524 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:55.163149 14559 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.178184 14372 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:55.176368 14522 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:55.299842 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 210995us
I20260709 13:15:55.172950 14477 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:55.175789 14472 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:55.291801 14441 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.297115 14371 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:55.292815 14568 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:55.295928 14376 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:55.296941 14482 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:55.297446 14524 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:55.312317 14372 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.317505 14522 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.183229 14223 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:55.353839 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 240832us
I20260709 13:15:55.313446 14531 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:55.343250 14581 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:15:55.333721 14441 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.344599 14570 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:55.321384 13844 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:55.303792 14489 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:55.519038 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 358775us
I20260709 13:15:55.520509 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:55.522858 12032 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:15:55.343907 14578 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
W20260709 13:15:55.369386 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.339962 14482 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:55.387060 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.349673 14372 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:55.382170 14585 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:15:55.374647 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 238028us
I20260709 13:15:55.337461 14524 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.337863 14505 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:55.113852 14534 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:55.411284 14510 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.558s	user 0.001s	sys 0.009s
I20260709 13:15:55.364792 14538 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:55.426723 14584 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:15:55.426855 14591 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:15:55.435274 11367 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:55.435768 11515 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:15:55.414149 14583 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:15:55.331131 14418 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:55.362305 14317 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:55.302189 14466 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:55.527832 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 389911us
I20260709 13:15:55.586684 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:55.587683 14484 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:55.587999 14605 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.588521 14605 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:55.354714 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 252580us
I20260709 13:15:55.590487 14605 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:55.592097 14601 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:15:55.593227 14597 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:15:55.593714 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.594432 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 690) took 1920 ms. Trace:
I20260709 13:15:55.540977 14587 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:15:55.594808 11367 rpcz_store.cc:276] 0709 13:15:53.673726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.806963 (+133237us) service_pool.cc:224] Handling call
0709 13:15:53.807195 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:55.574881 (+1767686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.594160 (+ 19279us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b5400166054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:55.594352 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3743492352}
W20260709 13:15:55.537725 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 182199us
I20260709 13:15:55.601230 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:55.601884 11366 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:55.601930 12060 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:15:55.609093 11362 pending_rounds.cc:85] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 7
I20260709 13:15:55.609495 11362 pending_rounds.cc:107] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:15:55.609737 11362 raft_consensus.cc:2889] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:55.541496 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 391492us
I20260709 13:15:55.546263 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:55.471504 14284 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.550190 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:55.614667 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 64569.7us
I20260709 13:15:55.615331 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:55.556933 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 537850us
I20260709 13:15:55.557104 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:55.537302 14465 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:55.557962 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:55.558295 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:55.619879 12068 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:55.559193 12069 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:15:55.560170 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:55.416584 14590 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 13:15:55.540982 14481 consensus_queue.cc:1243] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:55.473598 14477 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:55.540901 13844 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:55.576182 12061 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:15:55.549623 14382 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:55.579931 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:55.562352 14511 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:55.581719 11268 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 7, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:55.636024 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 75951.3us
I20260709 13:15:55.635571 11268 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.582301 14241 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.639253 14607 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.639803 14607 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.640724 14241 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:55.641359 11363 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:55.641336 14522 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.641901 14607 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:55.623375 11333 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:55.625756 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:55.619968 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:55.593191 14601 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:15:55.618520 14578 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:55.637046 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:55.637944 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:55.638095 12065 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:55.637508 11268 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 6, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.636267 11409 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:15:55.636165 14596 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:55.649487 14605 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:55.624061 14538 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:55.667217 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 195141us
I20260709 13:15:55.653883 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.654243 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:55.654615 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.684552 11439 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:55.657241 11366 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:55.676119 11268 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 6, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.620106 14611 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.572011 14510 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:55.650156 14483 consensus_meta.cc:287] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.000s
I20260709 13:15:55.628418 14507 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:55.683650 11441 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:55.619007 14484 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:15:55.650094 14308 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:15:55.685283 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:55.687928 11266 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56588 (request call id 236) took 16243 ms (client timeout 15000 ms). Trace:
I20260709 13:15:55.693099 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.692373 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:55.629606 14604 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:55.700555 14223 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:55.762436 11268 catalog_manager.cc:5697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 3 to 7, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.681048 14570 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:55.805645 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 704) took 1068 ms. Trace:
I20260709 13:15:55.621120 14258 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:15:55.820331 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:55.651212 14601 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:55.703167 14508 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:55.772426 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 136306us
I20260709 13:15:55.649927 13844 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:55.763196 14579 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:15:55.833003 11517 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:15:55.681450 14531 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:55.828509 14573 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:55.845273 11268 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 4 to 5, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.701853 14353 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:55.840481 11369 rpcz_store.cc:276] 0709 13:15:54.737062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.854635 (+117573us) service_pool.cc:224] Handling call
0709 13:15:54.854963 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:54.870628 (+ 15665us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:54.890557 (+ 19929us) log.cc:844] Serialized 14 byte log entry
0709 13:15:55.056072 (+165515us) log.cc:844] Serialized 14 byte log entry
0709 13:15:55.056364 (+   292us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:55.056834 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:15:55.056986 (+   152us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:55.057022 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.057091 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:55.348184 (+291093us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18001f94e0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f
0709 13:15:55.411955 (+ 63771us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b18001f94e0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:55.399394 (+-12561us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b4800002958. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f 00007f35c1a15f37 00007f35c1a2a55c 00007f35c1a2a508 00007f35c1a2a488 00007f35c1a2a443 00007f35c1a296ec 00007f35c0953a96
0709 13:15:55.410542 (+ 11148us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b18001f94e0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:55.475876 (+ 65334us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b18001f94e0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911557 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:55.542011 (+ 66135us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b18001f94e0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911557 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:55.804583 (+262572us) raft_consensus.cc:1685] finished
0709 13:15:55.804605 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:55.805562 (+   957us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34795,"prepare.queue_time_us":23091,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":21740,"raft.run_cpu_time_us":3812,"raft.run_wall_time_us":362155,"spinlock_wait_cycles":84218880,"thread_start_us":139052,"threads_started":1,"wal-append.queue_time_us":219007}
I20260709 13:15:55.852916 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:55.885381 11409 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:55.647163 14522 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:55.644651 14439 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.812508 14308 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:55.806571 11266 rpcz_store.cc:269] 0709 13:15:39.444205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.478274 (+ 34069us) service_pool.cc:224] Handling call
0709 13:15:55.687837 (+16209563us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:53.034573 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:53.179935 (+145362us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:53.179996 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:15:53.180010 (+    14us) tablet.cc:662] Decoding operations
0709 13:15:53.181240 (+  1230us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:53.181315 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:53.181344 (+    29us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:15:53.182488 (+  1144us) tablet.cc:701] Row locks acquired
0709 13:15:53.182535 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:15:53.182708 (+   173us) write_op.cc:270] Start()
0709 13:15:53.182856 (+   148us) write_op.cc:276] Timestamp: P: 1783602953182641 usec, L: 0
0709 13:15:53.182876 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:15:53.298425 (+115549us) log.cc:844] Serialized 4739 byte log entry
0709 13:15:53.907371 (+608946us) op_driver.cc:464] REPLICATION: finished
0709 13:15:54.595558 (+688187us) write_op.cc:301] APPLY: starting
0709 13:15:54.595658 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:54.595936 (+   278us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:54.596073 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:54.694504 (+ 98431us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:54.694807 (+   303us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:15:54.694850 (+    43us) write_op.cc:312] APPLY: finished
0709 13:15:54.695188 (+   338us) log.cc:844] Serialized 89 byte log entry
0709 13:15:54.695372 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:54.704481 (+  9109us) write_op.cc:454] Released schema lock
0709 13:15:54.704547 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:54.704930 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:54.704959 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":932550,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":645967,"apply.run_cpu_time_us":4521,"apply.run_wall_time_us":109385,"num_ops":12,"prepare.queue_time_us":432285,"prepare.run_cpu_time_us":11303,"prepare.run_wall_time_us":603603,"raft.queue_time_us":184140,"raft.run_cpu_time_us":29213,"raft.run_wall_time_us":628909,"replication_time_us":724317,"thread_start_us":1136455,"threads_started":4,"wal-append.queue_time_us":248306,"wal-append.run_cpu_time_us":8492,"wal-append.run_wall_time_us":2121450}]]}
I20260709 13:15:55.838418 12061 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:55.843672 11364 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:55.685120 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:55.913586 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 228535us
I20260709 13:15:55.773221 14483 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:55.863257 14472 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.542s	user 0.004s	sys 0.005s
I20260709 13:15:55.812070 14507 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.918699 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:55.856479 14570 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:55.761390 14611 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.942683 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:55.943744 11519 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:15:55.944330 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 57887.7us
I20260709 13:15:55.945261 11517 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:55.872149 11268 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 3 to 5, now has a pending config: VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65), VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66), VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67), VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:55.918409 14308 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:55.973748 11268 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 3 to 6, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.859984 14599 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:55.970221 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.945957 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:55.977272 11514 raft_consensus.cc:2410] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:15:55.969787 11512 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:55.890552 14382 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:55.978616 14382 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:55.911657 14607 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:15:55.867643 14352 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.806s	user 0.006s	sys 0.004s
I20260709 13:15:55.981699 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:55.979030 14353 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:56.005115 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:55.923899 14538 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:55.938076 14472 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:56.012070 11366 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:56.013638 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:55.920027 14606 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:56.024642 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:56.026991 11367 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:55.945438 14604 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:56.024923 11370 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:55.918761 14595 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
W20260709 13:15:56.031411 11409 connection.cc:256] RPC call timeout handler was delayed by -1.3369063249999726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 13:15:56.031991 12061 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:15:56.034484 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:56.041024 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:56.027604 11512 pending_rounds.cc:85] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 6
I20260709 13:15:56.046113 11512 pending_rounds.cc:107] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:15:56.046387 11512 raft_consensus.cc:2889] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:56.047896 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:56.048666 12069 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:56.041741 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 58853.7us
I20260709 13:15:56.007469 11268 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:56.053440 11268 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:15:56.055202 11268 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:15:56.057999 12069 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:56.059867 11268 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:56.009430 14621 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.068440 14621 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:56.070626 14621 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:56.063055 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:56.004077 14607 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:56.072232 14607 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:56.076886 14616 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:56.078991 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:56.079630 11445 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:15:56.080897 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:56.082933 14616 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:56.083292 14616 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:56.083734 14616 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:56.090678 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:55.918792 14608 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.153s
I20260709 13:15:56.026772 14606 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:56.096283 11268 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 7, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:56.011387 14352 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:56.103974 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:56.126953 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.127780 12068 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
W20260709 13:15:56.127599 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 454164us
I20260709 13:15:56.061714 14477 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:56.134261 11268 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 6, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:56.145916 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 492203us
I20260709 13:15:56.147346 14258 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:15:56.148007 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54614.1us
I20260709 13:15:56.148417 14531 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:15:56.149933 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.152072 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:56.153255 14508 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.153549 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:56.153813 14508 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:56.154311 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:56.154496 14505 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:56.153273 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:56.154978 14505 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:56.155213 11511 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:56.156423 11514 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:56.156915 14505 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:56.157563 14508 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:56.157608 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:56.154654 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:56.159724 14317 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:56.162140 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:56.162251 11333 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:56.163158 14317 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:56.164173 14508 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:56.164656 14508 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.165098 14508 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:56.167303 14258 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:56.153002 14616 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:56.175127 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:56.176143 11517 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:56.176674 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:56.183616 11268 catalog_manager.cc:5697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 1 to 3, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 3 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:56.105540 14606 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:56.135257 14613 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
W20260709 13:15:56.189894 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 59488.8us
I20260709 13:15:56.099557 14573 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:15:56.185444 11268 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 2 to 4, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 4 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:56.194749 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45336.4us
I20260709 13:15:56.197795 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:56.098608 14607 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:56.193542 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:56.207721 14466 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:56.238916 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:56.238997 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:56.240736 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:56.241878 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:56.242463 12069 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:56.243523 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:56.200412 14610 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
I20260709 13:15:56.188170 14510 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:56.247359 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 54257.7us
I20260709 13:15:56.248082 11436 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:56.249158 11439 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:15:56.250185 11443 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:15:56.250972 14624 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:56.251097 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54526.3us
W20260709 13:15:56.209939 14371 consensus_meta.cc:287] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.870s	user 0.008s	sys 0.001s
W20260709 13:15:56.260445 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 67073.9us
I20260709 13:15:56.263850 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:56.210939 14505 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.292062 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:56.131415 14570 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:56.255836 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:56.253008 13913 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
W20260709 13:15:56.373646 14625 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:15:56.597069 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:56.841120 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 585332us
I20260709 13:15:56.296303 14483 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:56.291324 14399 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:15:56.626798 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.004s	sys 0.004s
W20260709 13:15:56.937393 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.007s	sys 0.004s
I20260709 13:15:56.337513 12069 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
W20260709 13:15:56.369239 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 66961.8us
W20260709 13:15:56.845381 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.941052 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:56.839026 14372 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.277s	user 0.008s	sys 0.002s
W20260709 13:15:56.626469 11510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.004s	sys 0.003s
I20260709 13:15:56.871150 12061 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:15:56.942432 11510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.008s	sys 0.003s
I20260709 13:15:56.874882 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:15:56.875855 11444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.010s	sys 0.036s
W20260709 13:15:56.943476 11444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.011s	sys 0.036s
W20260709 13:15:56.839864 14572 consensus_meta.cc:287] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.661s	user 0.007s	sys 0.006s
W20260709 13:15:56.838249 11518 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.007s	sys 0.003s
W20260709 13:15:56.944499 11518 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.007s	sys 0.003s
I20260709 13:15:56.291898 14439 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:56.397212 14559 consensus_meta.cc:287] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.082s	user 0.010s	sys 0.001s
W20260709 13:15:56.581388 14308 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.000s
W20260709 13:15:56.630475 14441 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.219s	user 0.010s	sys 0.003s
W20260709 13:15:56.946291 12066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.014s	sys 0.022s
W20260709 13:15:56.974548 12066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.014s	sys 0.022s
W20260709 13:15:56.585968 14314 consensus_meta.cc:287] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 1.292s	user 0.007s	sys 0.002s
W20260709 13:15:56.409144 14626 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
W20260709 13:15:56.960402 12035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.005s	sys 0.013s
W20260709 13:15:56.983043 12035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.010s	sys 0.013s
W20260709 13:15:56.578342 14645 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:15:56.839964 14635 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.340366 14466 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:56.407584 14619 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.382s	user 0.001s	sys 0.000s
W20260709 13:15:56.898828 14482 consensus_meta.cc:287] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.335s	user 0.007s	sys 0.004s
W20260709 13:15:56.405936 14623 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
W20260709 13:15:56.377058 14620 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:15:56.578720 14352 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:15:56.590052 14524 consensus_meta.cc:287] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.020s	user 0.008s	sys 0.000s
I20260709 13:15:56.403002 14630 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:56.569547 14611 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:56.376295 14631 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
I20260709 13:15:56.294598 14624 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:56.312903 14371 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:56.335618 14510 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:56.583722 14507 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:56.892598 14604 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.408705 14607 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:56.369700 14376 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.409945 14258 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:56.976963 13844 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.001s	sys 0.013s
W20260709 13:15:56.959627 14615 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.000s	sys 0.015s
I20260709 13:15:56.322574 14605 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.657s
I20260709 13:15:56.995074 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:57.041033 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46083.8us
I20260709 13:15:57.042325 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:56.868119 14638 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:57.010694 14602 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:15:56.953907 14399 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.977422 14572 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.056986 14399 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:57.038561 14507 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:57.072628 14507 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.074741 14507 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:57.075030 14633 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:56.384413 14636 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.655s
I20260709 13:15:56.963471 14483 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:57.015753 14648 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:56.968966 14372 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:56.984169 14308 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.003888 14635 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.017182 14418 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:57.007087 14466 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.019327 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 682) took 1201 ms. Trace:
I20260709 13:15:57.161108 11515 rpcz_store.cc:276] 0709 13:15:55.818381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.819494 (+  1113us) service_pool.cc:224] Handling call
0709 13:15:55.819836 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:56.198790 (+378954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:56.198856 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:56.199719 (+   863us) log.cc:844] Serialized 26 byte log entry
0709 13:15:56.502136 (+302417us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:56.627527 (+125391us) log.cc:844] Serialized 14 byte log entry
0709 13:15:56.843134 (+215607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.843228 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.019104 (+175876us) raft_consensus.cc:1685] finished
0709 13:15:57.019126 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.019264 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33979,"prepare.queue_time_us":125114,"prepare.run_wall_time_us":53,"raft.queue_time_us":34337,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":11281,"spinlock_wait_cycles":3124992,"thread_start_us":572198,"threads_started":2,"wal-append.queue_time_us":781300}
W20260709 13:15:57.005411 14606 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.804s	user 0.006s	sys 0.003s
I20260709 13:15:56.998592 14439 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.018354 14213 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:57.012247 14658 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.976586 14559 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:56.982326 14314 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:56.983893 14441 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.036582 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 712) took 1429 ms. Trace:
I20260709 13:15:57.036582 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 715) took 1428 ms. Trace:
I20260709 13:15:57.254025 11365 rpcz_store.cc:276] 0709 13:15:55.608419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.611044 (+  2625us) service_pool.cc:224] Handling call
0709 13:15:55.611537 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:55.611886 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:55.611913 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:55.621478 (+  9565us) log.cc:844] Serialized 26 byte log entry
0709 13:15:55.802953 (+181475us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:55.803005 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.803097 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.032959 (+1229862us) raft_consensus.cc:1685] finished
0709 13:15:57.032981 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.036512 (+  3531us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36730,"raft.queue_time_us":427,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":15542,"thread_start_us":107662,"threads_started":1,"wal-append.queue_time_us":202349}
I20260709 13:15:56.405591 14508 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.040011 14510 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.044562 14657 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:57.039388 14376 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:57.040359 13844 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.001s	sys 0.013s
I20260709 13:15:57.047883 14638 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:15:57.040593 14615 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.002s	sys 0.019s
I20260709 13:15:57.048470 14258 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.048974 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1065) took 1430 ms. Trace:
I20260709 13:15:57.054088 14599 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:56.321278 14505 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.076658 14624 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.026327 14630 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:57.088063 11268 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.036s	sys 0.043s
I20260709 13:15:57.037976 14604 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.078718 14572 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.012275 14482 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.023805 14524 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.122354 14648 consensus_queue.cc:1243] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:57.192809 11517 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:57.246995 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 820) took 1009 ms. Trace:
I20260709 13:15:57.301836 14599 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:57.252803 11368 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:15:57.259249 11437 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:57.260332 12062 raft_consensus.cc:2368] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:15:57.125221 14483 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.261529 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:15:57.262586 11483 connection.cc:256] RPC call timeout handler was delayed by -1.0346366680000756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:15:57.420492 11365 raft_consensus.cc:2368] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8.
I20260709 13:15:57.265053 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:57.420843 11444 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:57.421607 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.265267 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.420914 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1669) took 1380 ms. Trace:
I20260709 13:15:57.177546 14606 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.256405 11362 rpcz_store.cc:276] 0709 13:15:55.607591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.608569 (+   978us) service_pool.cc:224] Handling call
0709 13:15:55.608901 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:55.610238 (+  1337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:55.610268 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:55.610699 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:15:55.858031 (+247332us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:55.858066 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.858186 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.034332 (+1176146us) raft_consensus.cc:1685] finished
0709 13:15:57.034353 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.036511 (+  2158us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":247053,"threads_started":1,"wal-append.queue_time_us":324575}
I20260709 13:15:57.434507 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:57.179745 14439 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:57.269814 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 320411us
I20260709 13:15:57.434533 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:57.437419 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.188941 14213 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.295734 12065 rpcz_store.cc:276] 0709 13:15:55.618262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.620734 (+  2472us) service_pool.cc:224] Handling call
0709 13:15:55.637861 (+ 17127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:55.679480 (+ 41619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.048675 (+1369195us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54001fb1a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc975dc0 00007f35bca01a02 00007f35bc9fddd6 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:15:57.048910 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506005376}
I20260709 13:15:57.445920 11515 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:15:57.193442 14658 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.262307 14657 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:57.312961 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:57.322772 11268 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.041s	sys 0.053s
I20260709 13:15:57.267633 14605 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:57.335157 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1552) took 1785 ms. Trace:
W20260709 13:15:57.355970 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 481859us
I20260709 13:15:57.165271 14633 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:57.422361 11365 rpcz_store.cc:276] 0709 13:15:56.040771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.261240 (+1220469us) service_pool.cc:224] Handling call
0709 13:15:57.420847 (+159607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.421932 11361 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:57.474880 14534 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.474517 14371 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.423184 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:57.424466 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:57.425899 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:57.622725 11444 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:57.358418 12066 rpcz_store.cc:276] 0709 13:15:56.237995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.238092 (+    97us) service_pool.cc:224] Handling call
0709 13:15:56.238380 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:56.289626 (+ 51246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:56.289657 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:56.291253 (+  1596us) log.cc:844] Serialized 26 byte log entry
0709 13:15:56.975093 (+683840us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:56.975142 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.977592 (+  2450us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.246721 (+269129us) raft_consensus.cc:1685] finished
0709 13:15:57.246749 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.246927 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91553,"raft.queue_time_us":74660,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":17334,"spinlock_wait_cycles":1222912,"thread_start_us":657059,"threads_started":1,"wal-append.queue_time_us":805098}
I20260709 13:15:57.325613 14615 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:57.438472 11436 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:15:57.441020 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:57.441743 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:57.442467 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:57.447572 11442 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:57.466416 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 201200us
I20260709 13:15:57.356138 14359 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:15:57.356671 13844 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:15:57.471902 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 429621us
I20260709 13:15:57.436136 14572 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:57.436327 14353 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:57.449963 14669 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:57.448145 14570 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:57.599880 14466 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.443997 14507 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:57.598529 14601 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.470943 14657 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:57.620872 11510 rpcz_store.cc:276] 0709 13:15:55.549680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.550754 (+  1074us) service_pool.cc:224] Handling call
0709 13:15:55.555596 (+  4842us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:55.563738 (+  8142us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:15:55.623913 (+ 60175us) log.cc:844] Serialized 14 byte log entry
0709 13:15:56.942802 (+1318889us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:56.943455 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 13:15:56.943592 (+   137us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:56.943775 (+   183us) log.cc:844] Serialized 14 byte log entry
0709 13:15:57.052665 (+108890us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b1800168d00. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:57.265908 (+213243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.265979 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.334923 (+ 68944us) raft_consensus.cc:1685] finished
0709 13:15:57.334952 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.335098 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61790,"prepare.queue_time_us":190993,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"raft.queue_time_us":170089,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":40576,"spinlock_wait_cycles":27890304,"thread_start_us":1551850,"threads_started":3,"wal-append.queue_time_us":552497}
I20260709 13:15:57.630913 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:57.631115 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:57.638579 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 173597us
I20260709 13:15:57.618084 14607 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:57.665081 14572 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.492406 14663 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.178s
I20260709 13:15:57.617785 14665 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:57.649176 14534 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:57.682777 11444 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:57.745565 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:57.671196 11365 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
W20260709 13:15:57.665692 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 44431.7us
I20260709 13:15:57.694900 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:57.786265 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 348900us
I20260709 13:15:57.665640 14652 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:57.670243 14507 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.793437 14652 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.795620 14652 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.671717 14314 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.678301 14441 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.790208 11361 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:57.670822 14674 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.670066 14466 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 13:15:57.766683 11438 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:57.739820 14308 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:57.743420 14534 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:57.797500 14674 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.669941 14353 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.667694 14627 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.808856 14507 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:57.815623 11436 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:57.665956 14642 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.492060 14638 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.670192 14481 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.825486 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.784076 14666 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.668677 13844 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:57.815865 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 161432us
I20260709 13:15:57.832149 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:57.804615 14534 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:57.648844 14359 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.807354 14601 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:57.864614 14359 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.865376 11438 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:57.838001 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 185569us
W20260709 13:15:57.841929 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 213320us
I20260709 13:15:57.869436 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.870249 14662 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:57.870841 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.871589 14607 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:57.871775 14662 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:57.643500 14372 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.876708 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 633) took 1004 ms. Trace:
I20260709 13:15:57.810616 14648 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:57.670133 14570 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.810637 14665 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:15:57.888154 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 62761.7us
I20260709 13:15:57.671727 14559 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.875049 14481 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.891326 14601 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.892208 14601 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.892829 14563 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:57.893240 14563 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:57.881820 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 132429us
I20260709 13:15:57.807090 14667 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:57.894958 14601 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.895515 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1665) took 1881 ms. Trace:
I20260709 13:15:57.818686 14508 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.890182 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:57.895900 11368 rpcz_store.cc:276] 0709 13:15:56.014068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.014202 (+   134us) service_pool.cc:224] Handling call
0709 13:15:56.014504 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:57.373430 (+1358926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.895225 (+521795us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540035be54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:57.895449 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2847252864}
I20260709 13:15:57.897140 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:57.897676 11368 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:57.897838 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 713) took 2290 ms. Trace:
I20260709 13:15:57.898188 11363 rpcz_store.cc:276] 0709 13:15:55.607895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.608011 (+   116us) service_pool.cc:224] Handling call
0709 13:15:55.608294 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:55.641958 (+ 33664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.897592 (+2255634us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b5400423e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:57.897759 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75550464}
I20260709 13:15:57.637627 14605 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.899197 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:57.899634 14371 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:57.900213 14371 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:57.900687 14605 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.877090 12063 rpcz_store.cc:276] 0709 13:15:56.871859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.871963 (+   104us) service_pool.cc:224] Handling call
0709 13:15:56.872229 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:56.872433 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:57.302826 (+430393us) log.cc:844] Serialized 14 byte log entry
0709 13:15:57.876384 (+573558us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:57.876436 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:57.876483 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.876547 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.876653 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199484,"prepare.queue_time_us":619351,"prepare.run_cpu_time_us":4456,"prepare.run_wall_time_us":24791,"raft.queue_time_us":254979,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":19101,"thread_start_us":755908,"threads_started":3}
I20260709 13:15:57.901818 14678 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:57.902240 14601 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.902838 14678 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:57.902695 14601 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.903215 14678 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:57.904111 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:57.904011 14678 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.904666 11367 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:57.901525 14690 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:15:57.905217 14371 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.905853 14678 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:57.907829 14677 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:15:57.909276 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:57.901984 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.646883 14615 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:57.847941 14399 consensus_meta.cc:287] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.773s	user 0.007s	sys 0.003s
W20260709 13:15:57.952842 11517 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 1976) took 1744 ms (client timeout 1568 ms). Trace:
I20260709 13:15:57.952998 14399 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:57.954146 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 65063.1us
W20260709 13:15:57.912976 12062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1057) took 2363 ms (client timeout 1955 ms). Trace:
I20260709 13:15:57.891610 14353 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.923909 14524 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.831386 14510 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.640376 14633 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.958104 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.925290 14684 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:57.677599 14669 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:57.941656 11364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2198) took 3258 ms (client timeout 1605 ms). Trace:
W20260709 13:15:57.961380 11364 rpcz_store.cc:269] 0709 13:15:54.683516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.687646 (+  4130us) service_pool.cc:224] Handling call
0709 13:15:57.921270 (+3233624us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400165154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:57.941581 (+ 20311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2429953920}
I20260709 13:15:57.964989 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:57.965682 11440 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:15:57.944020 11438 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:57.947355 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:57.966816 11441 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:57.967202 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:57.899916 11363 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:57.953326 11517 rpcz_store.cc:269] 0709 13:15:56.196489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.197518 (+  1029us) service_pool.cc:224] Handling call
0709 13:15:57.915696 (+1718178us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54000e8154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:57.941258 (+ 25562us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2275072256}
I20260709 13:15:57.952414 11367 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:57.970860 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2205) took 1032 ms. Trace:
I20260709 13:15:57.972776 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:57.971247 11367 rpcz_store.cc:276] 0709 13:15:56.938755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.903876 (+965121us) service_pool.cc:224] Handling call
0709 13:15:57.970675 (+ 66799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.848176 14308 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:57.954948 12062 rpcz_store.cc:269] 0709 13:15:55.545136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.545274 (+   138us) service_pool.cc:224] Handling call
0709 13:15:57.908655 (+2363381us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b5400654fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:57.908794 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3757079424}
I20260709 13:15:57.976027 11368 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:57.942478 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:57.823184 14627 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:57.956552 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 62594.6us
I20260709 13:15:57.979987 11370 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:57.880017 14505 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.981643 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:57.950553 14607 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:57.892113 14481 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:57.948000 14483 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:57.995244 11436 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:15:57.909843 14258 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:57.991070 12064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.001s	sys 0.009s
I20260709 13:15:57.837607 14642 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.848839 14507 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
I20260709 13:15:57.970153 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.957552 14359 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:57.894089 14563 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:57.879751 14666 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.927203 14648 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:57.948375 14665 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:58.051241 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 233250us
I20260709 13:15:58.031231 11441 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:58.059199 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:58.004494 12064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.001s	sys 0.009s
I20260709 13:15:58.041837 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:58.137208 14418 consensus_meta.cc:287] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.973s	user 0.010s	sys 0.000s
W20260709 13:15:58.050132 14251 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.006s	sys 0.019s
I20260709 13:15:58.069360 14658 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.000828 14439 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:58.140014 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 169936us
I20260709 13:15:58.149255 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:57.969034 14353 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:58.103394 14663 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.971978 14510 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:15:58.058830 14682 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:58.109665 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:58.084537 14669 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:58.146813 11485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.006s	sys 0.026s
I20260709 13:15:57.959758 14604 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.010542 14606 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:58.149593 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.010529 14482 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.167475 11364 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:15:58.145226 14251 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.007s	sys 0.021s
I20260709 13:15:57.973212 14466 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.171s
I20260709 13:15:58.124745 14690 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.079361 14615 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.106225 14601 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.154170 14627 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:58.154320 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:58.041366 14679 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.246s
I20260709 13:15:58.158061 14695 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:58.161892 11485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.006s	sys 0.027s
I20260709 13:15:58.144428 14662 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.141103 14594 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:57.999519 14308 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.146718 14694 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:58.169777 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 60179.6us
W20260709 13:15:58.170022 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 211773us
I20260709 13:15:58.172577 14510 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:58.160259 14678 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:58.175025 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1667) took 2139 ms. Trace:
I20260709 13:15:58.181046 11334 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:58.141451 14359 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.183755 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:58.158962 14663 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:58.196838 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.158907 14353 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.195014 14466 consensus_queue.cc:1243] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:58.185508 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:58.177994 13844 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:58.199266 14665 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.185021 14601 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:58.201774 14642 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:58.203061 14599 consensus_meta.cc:287] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.754s	user 0.008s	sys 0.000s
I20260709 13:15:58.174741 14258 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:58.213730 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:58.227746 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:15:58.228360 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 126964us
I20260709 13:15:58.174991 14657 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.187369 14510 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.223927 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.203603 14701 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:58.204577 14371 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:58.235991 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.215898 14399 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:58.218773 14658 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.187731 14534 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:58.233700 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 85690.6us
I20260709 13:15:58.212579 14667 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.212412 14572 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.232851 11517 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:58.221740 14604 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.218194 14678 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:58.231714 14308 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:58.185781 14662 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.240640 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:58.236877 11511 raft_consensus.cc:2410] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:15:58.248454 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.216133 14674 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:58.175777 14690 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.234633 14594 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:58.234230 14615 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.236574 14251 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:58.237426 14510 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.246560 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.237535 14258 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.248829 11364 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:58.249481 11517 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:58.244172 14371 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:58.253923 11514 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.221719 14687 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.235502 14599 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.255638 14666 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.249408 14308 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.257623 14534 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:58.257867 11361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2213) took 1265 ms (client timeout 1673 ms). Trace:
I20260709 13:15:58.259159 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.259794 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.243607 14658 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.262133 11443 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:58.245028 14609 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:58.263891 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.197681 14694 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.273195 11517 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:58.273345 14648 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:58.253535 14678 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:58.268744 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.278988 11515 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.265971 14594 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:58.271670 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:58.264973 14251 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:58.272188 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.289759 11512 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:58.246939 14604 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:58.270768 11361 rpcz_store.cc:269] 0709 13:15:56.992502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.630649 (+638147us) service_pool.cc:224] Handling call
0709 13:15:58.257649 (+627000us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000a78d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:58.257803 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274497280}
I20260709 13:15:58.274082 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.290992 11516 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.291244 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.291874 11361 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:58.185950 14695 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.292798 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:58.293910 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:58.294661 14695 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.288115 14678 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.288347 14353 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.297847 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2329) took 3333 ms. Trace:
I20260709 13:15:58.298226 11437 rpcz_store.cc:276] 0709 13:15:54.964803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.107393 (+142590us) service_pool.cc:224] Handling call
0709 13:15:55.159869 (+ 52476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.373702 (+2213833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.297614 (+923912us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b54002099d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:58.297791 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4827754240}
I20260709 13:15:58.253926 14615 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.235947 14701 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:15:58.283735 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.245042 14667 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.305948 14371 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.267174 14466 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.268180 14687 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.310989 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:58.304641 11516 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:15:58.282948 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.274544 14609 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:58.313163 14695 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:58.312650 11514 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:58.308472 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.314191 14687 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.314574 11512 raft_consensus.cc:2410] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:15:58.305320 14657 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.313719 14695 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.320567 11517 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:15:58.271359 14308 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.321122 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.322374 14695 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.325143 14510 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.327088 14510 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.328243 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.329062 11361 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:58.329999 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.330683 11361 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:58.331744 11361 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:58.332321 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1668) took 2291 ms. Trace:
I20260709 13:15:58.332623 11361 rpcz_store.cc:276] 0709 13:15:56.040494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.331242 (+2290748us) service_pool.cc:224] Handling call
0709 13:15:58.331547 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.332122 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.332279 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.319720 11364 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:15:58.338064 11517 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:58.341517 11515 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:58.321847 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.344937 14510 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:58.345471 14667 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:58.346159 14605 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.346652 14605 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.320104 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.348188 14247 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:58.348671 14605 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.353397 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:58.357786 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.358143 11436 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:58.359160 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.355809 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.360122 11441 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:58.360849 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:58.358381 11445 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:58.362246 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:58.362193 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.361078 14701 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:58.364074 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.364701 11361 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:15:58.365041 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.366137 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.365491 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2204) took 1427 ms. Trace:
I20260709 13:15:58.348598 14247 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:58.367266 11361 rpcz_store.cc:276] 0709 13:15:56.938442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.364202 (+1425760us) service_pool.cc:224] Handling call
0709 13:15:58.364496 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.365194 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.365383 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.313748 14609 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.371419 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.371507 14609 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.347420 14684 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:58.372318 14684 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:58.372570 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.373204 14684 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.368254 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.374322 14684 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.371652 14599 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.349571 14359 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:58.320621 14466 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.409300 14615 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.410416 14658 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:58.373672 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.391803 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.412170 11437 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:58.413017 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.392619 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:58.413933 11363 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.345932 14663 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:58.433422 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.434118 11437 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:58.434897 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.435837 11437 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:15:58.436710 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:58.437352 11437 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:58.315459 14594 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.431694 14595 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.361889 14251 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.431603 14615 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.435690 14605 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:58.493228 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.467440 14595 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.506585 11363 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:15:58.512471 11363 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:58.513268 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 716) took 1411 ms. Trace:
I20260709 13:15:58.513638 11363 rpcz_store.cc:276] 0709 13:15:57.101698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.511911 (+1410213us) service_pool.cc:224] Handling call
0709 13:15:58.512283 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.513017 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.513209 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.464990 14690 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.620378 11361 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:58.621197 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2207) took 1681 ms. Trace:
I20260709 13:15:58.621608 11361 rpcz_store.cc:276] 0709 13:15:56.939286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.486159 (+1546873us) service_pool.cc:224] Handling call
0709 13:15:58.620184 (+134025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.620942 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.621139 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.611905 14604 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.641446 11361 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:15:58.642750 11361 pending_rounds.cc:85] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 6
I20260709 13:15:58.643067 11361 pending_rounds.cc:107] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:15:58.643366 11361 raft_consensus.cc:2889] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:58.526232 14605 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.469498 14247 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.511283 14615 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.666312 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 718) took 1368 ms. Trace:
I20260709 13:15:58.639896 14595 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:58.668224 11361 rpcz_store.cc:276] 0709 13:15:57.298105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.622412 (+1324307us) service_pool.cc:224] Handling call
0709 13:15:58.641078 (+ 18666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.666018 (+ 24940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.666233 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.673966 14247 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:58.698391 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 331189us
I20260709 13:15:58.671428 14308 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.716647 14466 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.707324 14616 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.746186 14615 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.745455 14605 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.754103 14695 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.781315 11364 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:15:58.781366 11363 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:58.783975 11361 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:58.785022 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 719) took 1483 ms. Trace:
I20260709 13:15:58.788573 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:15:58.798683 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 721) took 1325 ms. Trace:
I20260709 13:15:58.799229 11368 rpcz_store.cc:276] 0709 13:15:57.473550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.680010 (+1206460us) service_pool.cc:224] Handling call
0709 13:15:58.680346 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.680561 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:58.786917 (+106356us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.797801 (+ 10884us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.798000 (+   199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.798057 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:58.798151 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.798257 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.798476 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4247,"thread_start_us":9537,"threads_started":1}
I20260709 13:15:58.785533 11361 rpcz_store.cc:276] 0709 13:15:57.301972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.673312 (+1371340us) service_pool.cc:224] Handling call
0709 13:15:58.782865 (+109553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.784525 (+  1660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.784884 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:58.804343 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.806641 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2112) took 1382 ms. Trace:
I20260709 13:15:58.808454 11361 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:58.807152 11443 rpcz_store.cc:276] 0709 13:15:57.424147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.424254 (+   107us) service_pool.cc:224] Handling call
0709 13:15:58.806151 (+1381897us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54001126d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:58.806401 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470231808}
W20260709 13:15:58.814112 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.816211 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2206) took 1877 ms. Trace:
I20260709 13:15:58.816767 11364 rpcz_store.cc:276] 0709 13:15:56.939032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.344736 (+1405704us) service_pool.cc:224] Handling call
0709 13:15:58.364048 (+ 19312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.782821 (+418773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.815695 (+ 32874us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54002dad54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:58.816088 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957120256}
I20260709 13:15:58.817960 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:58.818775 11364 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:58.818828 11366 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:58.825120 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 723) took 1194 ms. Trace:
I20260709 13:15:58.825634 11361 rpcz_store.cc:276] 0709 13:15:57.625766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.802431 (+1176665us) service_pool.cc:224] Handling call
0709 13:15:58.803124 (+   693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.809160 (+  6036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.820041 (+ 10881us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:58.830127 14609 consensus_queue.cc:1059] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:58.829774 14595 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:58.832336 14611 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:58.833091 14611 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:58.835937 14609 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:58.836812 14609 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:58.841368 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 687) took 3016 ms. Trace:
I20260709 13:15:58.842823 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:58.844576 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 587) took 1407 ms. Trace:
W20260709 13:15:58.844950 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.834686 14595 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:58.188733 11369 rpcz_store.cc:276] 0709 13:15:56.035576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.037918 (+  2342us) service_pool.cc:224] Handling call
0709 13:15:56.067601 (+ 29683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:56.067796 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:56.103602 (+ 35806us) log.cc:844] Serialized 14 byte log entry
0709 13:15:57.923198 (+1819596us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b18002a1b40. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e23a 00007f35bca08a2a
0709 13:15:58.174538 (+251340us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.174588 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:58.174634 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.174863 (+   229us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.174972 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":342,"prepare.queue_time_us":35841,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":42543,"raft.queue_time_us":2751,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3629,"spinlock_wait_cycles":19298944,"thread_start_us":227066,"threads_started":1,"wal-append.queue_time_us":2075074}
I20260709 13:15:58.854815 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:15:58.855309 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 632) took 1983 ms. Trace:
I20260709 13:15:58.856387 11369 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:15:58.846272 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 1978) took 1468 ms. Trace:
I20260709 13:15:58.847100 11442 rpcz_store.cc:276] 0709 13:15:57.437111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.437287 (+   176us) service_pool.cc:224] Handling call
0709 13:15:57.437569 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.627816 (+190247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.843809 (+1215993us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b5400438ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:58.844276 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22461696}
I20260709 13:15:58.836030 14615 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:58.857878 11519 rpcz_store.cc:276] 0709 13:15:57.377969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.378084 (+   115us) service_pool.cc:224] Handling call
0709 13:15:57.419101 (+ 41017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.615127 (+196026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:57.615158 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:57.615201 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:57.615240 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.845699 (+1230459us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b54003d8454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:58.845778 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.846054 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169360128}
I20260709 13:15:58.867517 14223 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.868535 14223 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.841938 11518 rpcz_store.cc:276] 0709 13:15:55.824905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.825964 (+  1059us) service_pool.cc:224] Handling call
0709 13:15:55.826210 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:56.006422 (+180212us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:56.944947 (+938525us) log.cc:844] Serialized 14 byte log entry
0709 13:15:57.310527 (+365580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:57.310584 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:57.310626 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.840754 (+1530128us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54001509d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:58.840823 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.841147 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48233,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":6471,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":20457,"spinlock_wait_cycles":447266432,"thread_start_us":1267577,"threads_started":2,"wal-append.queue_time_us":885392}
I20260709 13:15:58.887264 14223 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.906205 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 722) took 1285 ms. Trace:
I20260709 13:15:58.909658 11368 rpcz_store.cc:276] 0709 13:15:57.620997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.801410 (+1180413us) service_pool.cc:224] Handling call
0709 13:15:58.862787 (+ 61377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.871821 (+  9034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.871847 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:58.871889 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:58.905910 (+ 34021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.905989 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.906128 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:58.917449 11510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.012s	sys 0.008s
W20260709 13:15:58.918118 11510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.012s	sys 0.008s
I20260709 13:15:58.919076 14534 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:58.919764 14534 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.920315 14534 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:58.855831 12060 rpcz_store.cc:276] 0709 13:15:56.871477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.871596 (+   119us) service_pool.cc:224] Handling call
0709 13:15:56.963359 (+ 91763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:57.263944 (+300585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:57.264025 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:57.269232 (+  5207us) log.cc:844] Serialized 26 byte log entry
0709 13:15:57.637653 (+368421us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:57.967564 (+329911us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.853919 (+886355us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18000ed3a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:15:58.854436 (+   517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.854577 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:58.854649 (+    72us) raft_consensus.cc:1685] finished
0709 13:15:58.854794 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.854991 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7965,"prepare.queue_time_us":333719,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":52346,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"spinlock_wait_cycles":25728896,"thread_start_us":844305,"threads_started":3,"wal-append.queue_time_us":597811}
W20260709 13:15:58.925683 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 71360.5us
I20260709 13:15:58.926491 14353 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:15:58.907657 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 90010.1us
I20260709 13:15:58.909817 14615 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:15:58.911845 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 69381.6us
I20260709 13:15:58.846740 14611 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.940058 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.941303 11515 raft_consensus.cc:2368] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8.
I20260709 13:15:58.941694 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.942957 11513 raft_consensus.cc:2368] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10.
I20260709 13:15:58.943742 14721 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.944322 11518 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:58.944774 14721 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.945382 14616 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.943279 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.947144 11511 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:58.948406 14615 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.951865 14609 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.952479 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.950549 14507 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.954694 14507 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.956504 14721 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.957283 14507 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:58.941592 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.950846 11361 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:58.949708 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.967535 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:58.969857 11369 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:15:58.936769 14223 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:58.951210 14251 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:15:58.954123 14605 consensus_queue.cc:1175] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b1906ab37ba541f493936685a808c8a9" responder_term: 6 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: [5.7-5.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:58.972474 14630 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.972868 14251 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:58.973443 14251 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:58.946265 14616 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.927495 14353 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.976527 14616 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.976825 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:58.977689 11368 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:58.956444 14573 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:58.979120 14573 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:58.972529 14223 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:58.981688 14573 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.970124 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:58.987797 14570 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:58.938045 14727 log.cc:927] Time spent T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:15:58.953713 14609 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:58.998895 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:58.977723 14353 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:58.992594 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.000129 11442 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:58.993470 14684 consensus_meta.cc:287] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.608s	user 0.013s	sys 0.004s
I20260709 13:15:59.005240 14353 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:59.005906 14353 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.006435 14353 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:59.005353 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.016486 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.015865 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.036270 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.020965 11445 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:59.025840 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.056365 11512 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:59.038753 11443 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:59.059896 11515 raft_consensus.cc:2368] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6.
I20260709 13:15:59.061386 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:59.040233 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.062278 11510 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:59.063848 11513 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:59.064893 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.067088 14573 consensus_queue.cc:1059] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:59.068322 14734 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:59.068637 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:59.068858 14734 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:59.069450 11514 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:59.069696 14251 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:59.070480 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.070168 14734 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.072000 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.074592 14731 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:59.075436 14731 consensus_queue.cc:1243] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:59.076020 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.077257 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.078109 11515 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:59.067847 11519 raft_consensus.cc:2368] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:15:59.019089 14223 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:59.027081 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:15:59.068418 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.034003 14694 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.038036 14635 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.079358 11515 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:59.058761 14624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.027s	sys 0.029s
I20260709 13:15:59.090030 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.079809 14734 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.096280 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:59.096432 14376 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.096961 11512 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.099953 11513 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:15:59.100282 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 592) took 1660 ms. Trace:
I20260709 13:15:59.100711 11439 rpcz_store.cc:276] 0709 13:15:57.439785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.624299 (+184514us) service_pool.cc:224] Handling call
0709 13:15:57.624571 (+   272us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:57.624900 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:57.624923 (+    23us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:57.669387 (+ 44464us) log.cc:844] Serialized 52 byte log entry
0709 13:15:58.799090 (+1129703us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b18003e3e60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:15:58.924771 (+125681us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:58.925417 (+   646us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.925724 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.925847 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:59.099800 (+173953us) raft_consensus.cc:1685] finished
0709 13:15:59.099913 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.100099 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":910,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":1006,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":31453,"spinlock_wait_cycles":25548288,"thread_start_us":49176,"threads_started":1,"wal-append.queue_time_us":1427957}
I20260709 13:15:59.104338 14251 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.105647 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:59.106415 11441 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.109534 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.110224 11445 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:15:59.110718 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.110896 11443 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:59.110880 14633 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:59.090420 14624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.028s	sys 0.029s
I20260709 13:15:59.079212 14353 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:59.113850 11518 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:15:59.123364 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:59.100570 14563 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:59.127081 11439 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.123383 14223 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:59.128983 14223 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:59.129617 14662 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.130281 14223 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:59.131762 11511 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:59.136492 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:59.137228 11443 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:59.131287 14609 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:59.126538 14534 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.149031 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 724) took 1518 ms. Trace:
I20260709 13:15:59.152565 11441 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:15:59.154381 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 127426us
W20260709 13:15:59.150467 14685 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.165s	user 0.016s	sys 0.009s
W20260709 13:15:59.160461 14685 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.016s	sys 0.009s
I20260709 13:15:59.161347 11445 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:15:59.151039 11362 rpcz_store.cc:276] 0709 13:15:57.637998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.805760 (+1167762us) service_pool.cc:224] Handling call
0709 13:15:58.806265 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.806761 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:58.862274 (+ 55513us) log.cc:844] Serialized 14 byte log entry
0709 13:15:59.148577 (+286303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:59.148627 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:59.148670 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.148737 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.148850 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":158571,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":136,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":274850,"threads_started":2,"wal-append.queue_time_us":235126}
I20260709 13:15:59.170451 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:15:59.173244 14684 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:59.173861 14684 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:59.177021 14738 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:59.178251 14736 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:59.178326 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.178658 14736 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:59.179149 11442 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:59.179706 14736 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.180500 14353 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:59.181413 14695 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.161659 14609 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.189479 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.190513 14694 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:59.190904 14694 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.201071 11512 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:15:59.201109 11443 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:15:59.191300 14694 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.199676 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.170774 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:15:59.206609 11439 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:59.180428 14284 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.174897 14684 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:59.214901 11441 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:59.216508 13844 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:59.216699 14247 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:59.191195 14635 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:15:59.224331 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 635) took 2351 ms. Trace:
I20260709 13:15:59.226854 14359 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:59.217206 13844 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.227767 14465 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.228379 14609 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.202884 14353 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:15:59.228322 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 717) took 1935 ms. Trace:
I20260709 13:15:59.214229 14633 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:59.240657 14353 consensus_queue.cc:1243] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:59.236820 11363 rpcz_store.cc:276] 0709 13:15:57.292552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.618452 (+1325900us) service_pool.cc:224] Handling call
0709 13:15:58.618802 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:58.784524 (+165722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.228000 (+443476us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54000c6554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:59.228243 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372521216}
I20260709 13:15:59.249451 11335 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.251199 14465 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.218501 14695 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:15:59.253961 14465 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:15:59.254770 14675 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 1.295s	user 0.002s	sys 0.012s
I20260709 13:15:59.235884 14534 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.256623 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 628) took 3204 ms. Trace:
I20260709 13:15:59.257016 12068 rpcz_store.cc:276] 0709 13:15:56.056884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.056992 (+   108us) service_pool.cc:224] Handling call
0709 13:15:56.057267 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:56.130023 (+ 72756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.256102 (+3126079us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b540034b554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:15:59.256423 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161699712}
I20260709 13:15:59.263116 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:59.265278 11517 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:59.267760 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.268507 11518 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:15:59.272171 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.253881 14630 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:59.274524 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:15:59.229733 12064 rpcz_store.cc:276] 0709 13:15:56.873055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.873153 (+    98us) service_pool.cc:224] Handling call
0709 13:15:56.873398 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:56.873566 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:57.259600 (+386034us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.129198 (+869598us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.137232 (+  8034us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b180044f680. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:15:59.223521 (+1086289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:59.223642 (+   121us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:59.223746 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:59.223856 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.224063 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1217136,"prepare.queue_time_us":875994,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":206612,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":70151,"spinlock_wait_cycles":14533888,"thread_start_us":1165925,"threads_started":4,"wal-append.queue_time_us":634879,"wal-append.run_cpu_time_us":1930,"wal-append.run_wall_time_us":1279099}
I20260709 13:15:59.265224 14633 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:59.274396 14630 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:15:59.289764 14630 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.290838 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:15:59.291114 11438 raft_consensus.cc:2410] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:15:59.292236 14573 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:59.295229 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 1.08862e+06us
I20260709 13:15:59.294873 14573 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:59.296999 14737 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:59.297533 14737 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.297955 14737 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:59.297969 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:15:59.301180 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.291183 14630 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.293169 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.306408 11444 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:59.303069 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:59.306896 11441 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.310297 11439 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:59.307700 11514 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:15:59.308089 14740 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 13:15:59.302472 14251 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:59.320926 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.321601 11443 raft_consensus.cc:1217] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-12.10]   Dedup: 6.7->[8.8-12.10]
I20260709 13:15:59.322322 11443 pending_rounds.cc:85] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 7
I20260709 13:15:59.322573 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 725) took 1638 ms. Trace:
I20260709 13:15:59.323486 14247 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:59.324018 14247 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:59.324992 14247 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.326462 14213 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:59.324018 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.334306 14247 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:59.336764 14752 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:15:59.337610 13844 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.339854 14608 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:15:59.340498 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 341873us
I20260709 13:15:59.340822 14608 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:59.341840 14358 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:59.342486 14608 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:59.342949 14608 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:59.343756 14636 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:59.344893 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.345176 14633 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.345679 14652 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:59.346060 14352 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:59.346486 14352 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:59.346863 14652 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:59.347285 14652 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:15:59.347513 14352 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.349664 14736 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:59.351323 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.351783 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.352183 14690 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:59.352623 14690 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:59.353873 14638 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:15:59.353725 14690 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.354560 14706 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.355175 14706 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.356982 14635 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:59.357517 14706 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.354168 14589 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.322610 11443 pending_rounds.cc:107] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 9.8
W20260709 13:15:59.337757 14744 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:15:59.334815 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 64642.6us
W20260709 13:15:59.323520 14746 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:15:59.352200 12064 raft_consensus.cc:2368] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:15:59.319797 14251 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.369071 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:15:59.323076 11370 rpcz_store.cc:276] 0709 13:15:57.684625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.810129 (+1125504us) service_pool.cc:224] Handling call
0709 13:15:58.810810 (+   681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:58.811540 (+   730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:58.811568 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:58.812635 (+  1067us) log.cc:844] Serialized 26 byte log entry
0709 13:15:58.935989 (+123354us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:58.936027 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.936092 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:59.322240 (+386148us) raft_consensus.cc:1685] finished
0709 13:15:59.322262 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:59.322396 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34132,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":250,"thread_start_us":122859,"threads_started":1,"wal-append.queue_time_us":299904}
I20260709 13:15:59.367805 11443 raft_consensus.cc:2889] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:59.352396 12068 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:59.370124 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.354810 14690 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:59.373945 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:59.358127 14717 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.674s	user 0.000s	sys 0.002s
I20260709 13:15:59.375849 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.380303 11369 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:59.380283 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.380048 11333 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.386368 11517 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:15:59.345036 14636 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:15:59.373407 14251 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.376873 14635 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:59.394034 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 82901.6us
I20260709 13:15:59.394769 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:59.395923 14563 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:59.396430 14563 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.396442 14376 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:59.396973 14563 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.397035 14376 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:59.397521 14376 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.399211 14695 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:59.400550 14652 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.401990 14652 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.403741 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.399773 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:15:59.405344 14601 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:59.405844 14601 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:15:59.387548 14589 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.407994 14694 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.390164 14638 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.410300 14694 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.411037 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.421727 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:59.393769 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 59027.3us
I20260709 13:15:59.371975 14734 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:15:59.433836 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:59.434509 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.391521 14352 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:59.452055 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:59.452396 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.452636 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:15:59.406816 14601 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:59.425527 14738 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:15:59.480715 14738 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:15:59.484969 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 51517.4us
I20260709 13:15:59.485973 14371 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:59.486071 14738 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:59.494208 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 86014.1us
I20260709 13:15:59.491711 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.522652 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.489069 14694 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:59.532145 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:15:59.539584 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:15:59.521065 11483 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.673s (SENT) [suppressed 5 similar messages]
I20260709 13:15:59.521894 12060 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:59.571388 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:59.573412 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 52377.8us
I20260709 13:15:59.530412 14738 raft_consensus.cc:2764] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:15:59.587296 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 67564.9us
I20260709 13:15:59.456259 14608 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.592228 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.604178 11514 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:59.583210 14615 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:59.610119 11483 leader_election.cc:336] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.955s (SENT)
I20260709 13:15:59.632758 11364 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: [4.3-7.5]
I20260709 13:15:59.632814 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:15:59.571221 14371 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:15:59.591665 14679 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:15:59.646611 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.651605 11515 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:59.652573 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.653282 12066 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:59.653242 11441 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:15:59.655757 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:59.656782 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.682008 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.646034 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.699719 12065 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:59.705487 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.681046 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:59.706804 14762 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:15:59.611588 14638 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:15:59.572189 14635 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.709995 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.706805 11365 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
W20260709 13:15:59.712901 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 279333us
I20260709 13:15:59.710672 12063 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:15:59.723466 14734 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:59.724156 14734 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.726428 14734 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:59.734364 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.709075 14738 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:59.736279 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.740787 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.720265 14608 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.708086 14679 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:15:59.745824 14756 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 13:15:59.746627 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.746881 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.746987 14258 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:15:59.747831 14761 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:15:59.748389 11443 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:59.748860 14573 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.749734 14213 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:15:59.752166 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.765025 11514 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:59.709475 14615 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:15:59.768896 11362 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:15:59.770218 14760 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:15:59.798131 11445 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:15:59.771704 14605 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:59.772377 11443 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:15:59.773159 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.737756 14589 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.779872 11516 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:59.779661 14622 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 13:15:59.803960 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 202917us
I20260709 13:15:59.807240 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.807544 12066 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:59.808072 11365 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:15:59.808210 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.808337 14376 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.809617 11370 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:59.812523 14376 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.768338 14284 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:15:59.815233 14284 consensus_queue.cc:1243] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:15:59.814642 14615 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:15:59.818039 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2123) took 1919 ms. Trace:
I20260709 13:15:59.818473 11438 rpcz_store.cc:276] 0709 13:15:57.898887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.966900 (+ 68013us) service_pool.cc:224] Handling call
0709 13:15:59.817798 (+1850898us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003e49d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:15:59.817961 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042099968}
I20260709 13:15:59.819505 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.820156 12060 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:59.821182 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.822254 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:59.787537 14758 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:15:59.753827 14251 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.833256 14251 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.835035 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.836126 11367 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:59.836269 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.836833 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.837888 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.838841 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.836840 12060 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:59.840288 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.840986 12060 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:15:59.841635 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.842622 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.843914 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.845077 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.847225 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:15:59.848013 14737 consensus_meta.cc:287] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.001s
I20260709 13:15:59.848423 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.848558 14737 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.849454 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.807148 14213 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.831574 14738 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:59.855116 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:59.861064 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.837260 14679 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.787287 14563 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.863255 11445 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:15:59.863665 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.850409 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.868625 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:59.869316 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 66662.4us
I20260709 13:15:59.869648 14770 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:59.801554 14605 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:15:59.880900 12061 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 9. Ops: []
I20260709 13:15:59.882090 12066 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:15:59.882577 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.882496 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.882961 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.881865 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:15:59.878310 14736 consensus_meta.cc:287] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.528s	user 0.000s	sys 0.012s
I20260709 13:15:59.897186 11439 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:15:59.901806 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.896456 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.907975 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:15:59.908799 11514 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:59.909629 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 55127.2us
I20260709 13:15:59.910056 11517 pending_rounds.cc:85] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 3
I20260709 13:15:59.910387 11517 pending_rounds.cc:107] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:15:59.910656 11517 raft_consensus.cc:2889] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:59.911741 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:15:59.916724 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.917450 11366 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:15:59.905894 14738 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:59.920558 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.921054 14636 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:59.921886 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.936702 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.939800 14382 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:59.940141 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.940375 14700 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:59.941215 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.942251 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.942718 12060 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:59.930371 12068 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:15:59.896109 14563 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:15:59.926141 12069 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:59.909581 14358 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:59.964125 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.964548 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.966198 12066 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:15:59.946341 14652 consensus_meta.cc:287] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.000s
I20260709 13:15:59.970474 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:15:59.973577 12069 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:59.973899 11369 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
W20260709 13:15:59.976109 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.976456 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.976999 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.977703 11365 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 4: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:15:59.978201 11362 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:59.979490 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:15:59.976558 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.974448 12062 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:15:59.981793 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:15:59.983564 11364 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:15:59.985988 14737 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:15:59.935590 14605 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:15:59.988312 11514 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:15:59.990106 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:15:59.996191 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.000262 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.001091 12066 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:16:00.001945 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:00.004617 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:00.005810 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:00.008894 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:00.001292 14630 consensus_meta.cc:287] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.687s	user 0.005s	sys 0.005s
I20260709 13:16:00.010207 11362 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:00.011613 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.016171 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.014895 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.018391 11511 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:16:00.023746 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.024367 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:00.018838 11511 pending_rounds.cc:85] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 7
I20260709 13:16:00.026211 11511 pending_rounds.cc:107] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:16:00.026468 11511 raft_consensus.cc:2889] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:00.013990 14352 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:00.032502 11364 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:00.035137 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:00.032080 14352 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:00.038355 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.042932 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.044436 14247 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:00.044801 14247 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:00.045671 14247 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.047770 14247 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:00.032258 12063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:00.065920 11366 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:00.067257 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:00.067576 12066 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:00.077800 11362 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:00.086690 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:00.091691 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:00.094394 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54679.9us
I20260709 13:16:00.090075 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:00.097569 11366 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:00.107748 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:00.040920 14352 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.167603 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:00.168509 11445 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:00.170172 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 288378us
I20260709 13:16:00.174057 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:00.182037 11366 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:00.102843 14700 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:16:00.103000 14652 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:16:00.213032 11440 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:00.219265 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:00.262774 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 86255.6us
W20260709 13:16:00.273028 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 62784.5us
I20260709 13:16:00.273885 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.271533 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:00.283762 11363 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:00.286850 11484 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:00.294432 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:00.295059 11440 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:00.273916 14769 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:00.333676 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:00.336747 11445 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:00.267760 14700 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:16:00.287279 14630 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:00.298079 14608 consensus_meta.cc:287] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
W20260709 13:16:00.342540 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:00.354583 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:00.348328 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 208194us
W20260709 13:16:00.352355 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 49405.9us
I20260709 13:16:00.367653 14314 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:00.368232 14314 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:00.369216 14314 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.351048 14638 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:16:00.371625 14777 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:16:00.372815 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 214409us
I20260709 13:16:00.374526 14638 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:00.374977 14638 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:00.375484 11436 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:00.376860 11484 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:00.377094 11436 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:00.378476 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:00.379654 14738 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:00.384232 14700 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:00.385421 14652 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:00.386000 14652 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.388171 14652 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:00.388708 14652 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:00.389066 14652 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:00.389163 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.389894 14652 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.391103 14652 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:00.392017 14608 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:00.392901 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.293428 14685 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.350577 14736 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:00.351675 14630 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.370534 11362 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:00.403844 14783 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:00.422189 11440 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:00.423736 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:00.421260 12060 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:00.370635 14314 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:00.433375 11445 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:00.436022 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:00.437161 14769 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:00.445851 12064 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:00.437513 14685 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.448005 12068 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:00.440364 14783 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:00.441078 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:00.439657 14635 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:00.423167 14608 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:00.446259 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:00.426625 14736 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:00.432703 14382 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.441612 14630 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:00.460073 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:00.472388 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 253639us
W20260709 13:16:00.479324 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.488933 12068 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:00.442973 14601 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:16:00.465703 14736 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:00.500476 12065 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:00.504956 14775 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.495306 14784 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:00.498459 14783 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:00.469448 14635 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:00.473906 14382 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.467351 14638 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.517968 14608 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.529363 14608 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:00.530205 14738 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:00.531144 14213 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:00.517107 14775 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.532088 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:00.532644 14685 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:00.533528 14775 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:00.534250 14685 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:00.534451 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 601) took 1109 ms. Trace:
I20260709 13:16:00.534809 11439 rpcz_store.cc:276] 0709 13:15:59.425356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.519334 (+ 93978us) service_pool.cc:224] Handling call
0709 13:16:00.534263 (+1014929us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54002085d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:00.534370 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761104640}
I20260709 13:16:00.513423 14784 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.535681 14784 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:00.540154 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:00.543749 14775 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:00.544559 14609 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:00.545048 14609 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.545428 14609 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:00.518841 14783 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:00.554380 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:00.556815 11408 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.004s	sys 0.027s
W20260709 13:16:00.562048 11408 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.004s	sys 0.027s
W20260709 13:16:00.562330 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 585748us
W20260709 13:16:00.562875 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.544802 14775 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:00.587720 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.585536 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.588367 11514 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:00.534843 14685 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:00.571166 14782 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:16:00.641618 12069 consensus_meta.cc:287] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.657s	user 0.007s	sys 0.001s
I20260709 13:16:00.642071 12069 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:16:00.645048 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.645968 11518 raft_consensus.cc:2410] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:16:00.624377 14589 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.026s	sys 0.053s
W20260709 13:16:00.647970 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.648586 12066 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:00.587147 14775 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:00.559370 14783 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:00.631625 14636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.021s	sys 0.002s
I20260709 13:16:00.664878 14784 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:00.649544 14589 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.026s	sys 0.053s
I20260709 13:16:00.667321 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 602) took 1178 ms. Trace:
W20260709 13:16:00.658512 14636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.021s	sys 0.002s
I20260709 13:16:00.667774 11441 rpcz_store.cc:276] 0709 13:15:59.480728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.654621 (+173893us) service_pool.cc:224] Handling call
0709 13:15:59.654985 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:59.734080 (+ 79095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:59.734155 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:59.752751 (+ 18596us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.053707 (+300956us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:00.053750 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.053816 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:00.546812 (+492996us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:00.656176 (+109364us) raft_consensus.cc:1685] finished
0709 13:16:00.656208 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.659035 (+  2827us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73172,"raft.queue_time_us":82952,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":11408,"spinlock_wait_cycles":57689472,"thread_start_us":229433,"threads_started":1,"wal-append.queue_time_us":702743}
I20260709 13:16:00.655814 14738 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:00.708039 11439 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:16:00.713337 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:00.718492 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:00.702315 14636 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:00.731551 11437 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:00.736510 11437 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:00.744642 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:00.745541 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:00.746013 14792 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:16:00.746294 11513 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:00.747208 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.748181 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 733) took 1351 ms. Trace:
I20260709 13:16:00.748574 11363 rpcz_store.cc:276] 0709 13:15:59.387932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.389860 (+  1928us) service_pool.cc:224] Handling call
0709 13:15:59.401154 (+ 11294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:00.284261 (+883107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.738881 (+454620us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b540024a754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:00.739128 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2029352192}
I20260709 13:16:00.750044 11519 raft_consensus.cc:2410] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:16:00.750689 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:00.751645 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 48462.3us
I20260709 13:16:00.756263 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:00.757090 11369 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:00.759379 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.706666 14738 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.763595 11362 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:00.781564 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:00.748039 14652 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
W20260709 13:16:00.787955 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 341768us
I20260709 13:16:00.783727 11437 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.798053 14796 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:00.800019 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.798574 14796 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:00.816013 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 387609us
I20260709 13:16:00.750218 14636 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.739688 14775 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:00.818638 11362 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:00.825050 11440 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:16:00.834053 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.871937 11510 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:00.873656 14635 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:00.874372 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.874985 11519 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:00.875021 14695 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:00.876215 11519 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:00.877221 14695 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.877746 14695 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.878207 11519 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:00.816836 14630 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:00.879856 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:00.880679 14630 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:00.882567 14636 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:00.883150 11510 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:16:00.884145 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.885070 14695 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:00.886531 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:00.887825 11510 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:16:00.890345 14685 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:00.891857 14775 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:00.895804 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.896658 11519 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:16:00.891163 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 638) took 1073 ms. Trace:
I20260709 13:16:00.897500 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:00.897703 12062 rpcz_store.cc:276] 0709 13:15:59.818186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.061957 (+243771us) service_pool.cc:224] Handling call
0709 13:16:00.062992 (+  1035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:00.100425 (+ 37433us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:00.100725 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.425567 (+324842us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:00.495554 (+ 69987us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.495712 (+   158us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:00.495753 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.495818 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:00.890926 (+395108us) raft_consensus.cc:1685] finished
0709 13:16:00.890948 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.891099 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33669,"prepare.queue_time_us":12645,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":37878,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":15103,"spinlock_wait_cycles":24960,"thread_start_us":324066,"threads_started":1,"wal-append.queue_time_us":529285}
I20260709 13:16:00.898049 11519 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:00.900893 12063 raft_consensus.cc:606] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:00.901274 12063 raft_consensus.cc:614] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:00.880930 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:00.902411 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 73586.4us
I20260709 13:16:00.907080 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:00.915032 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:00.917173 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 853) took 1064 ms. Trace:
I20260709 13:16:00.918236 12060 rpcz_store.cc:276] 0709 13:15:59.852612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.439481 (+586869us) service_pool.cc:224] Handling call
0709 13:16:00.439761 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:00.439966 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:00.452495 (+ 12529us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.916829 (+464334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:00.916884 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:00.916930 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.916996 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.917114 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87006,"prepare.queue_time_us":9141,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"thread_start_us":349592,"threads_started":1}
W20260709 13:16:00.921077 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 110415us
W20260709 13:16:00.921888 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.920212 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:00.922405 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:00.912750 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 96017.4us
I20260709 13:16:00.890977 14609 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:00.924897 11518 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:00.925781 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:00.807551 14690 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 13:16:00.910486 14784 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:00.941140 11510 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:00.933259 14314 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:00.944495 14314 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:00.927517 14695 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:00.945676 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:00.957086 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:00.971097 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.941705 14213 raft_consensus.cc:2764] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:16:00.945999 14630 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:00.991348 11334 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:00.959488 14695 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:01.009266 11484 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.010s	sys 0.042s
W20260709 13:16:01.009585 11484 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.010s	sys 0.042s
W20260709 13:16:01.009836 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 568828us
I20260709 13:16:00.990370 14776 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.010509 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:00.997912 14784 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.957396 14635 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:16:01.010700 14695 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:01.021150 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 130611us
I20260709 13:16:01.008369 14795 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:01.028314 11436 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:01.018651 14784 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.032207 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:01.038517 11443 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:01.027845 14738 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.016008 14776 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.031085 14806 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:01.020219 14783 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.066313 14605 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:01.067829 11334 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.071069 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.087092 14352 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:01.089690 14801 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:16:01.092494 14635 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:01.094041 11334 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:01.095530 14810 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:01.095826 14635 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.095374 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.097496 14635 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:01.098342 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2020) took 1212 ms. Trace:
I20260709 13:16:01.099602 14816 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:01.098749 11515 rpcz_store.cc:276] 0709 13:15:59.886063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.886300 (+   237us) service_pool.cc:224] Handling call
0709 13:16:00.450250 (+563950us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b5800010b98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:16:00.450446 (+   196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:00.450792 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:00.450817 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:00.451262 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.905601 (+454339us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:00.905638 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.905700 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.098037 (+192337us) raft_consensus.cc:1685] finished
0709 13:16:01.098061 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.098281 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2726,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":11686,"spinlock_wait_cycles":3779968,"thread_start_us":453963,"threads_started":1,"wal-append.queue_time_us":584429}
I20260709 13:16:01.101042 14635 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:01.101627 14635 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.104655 14784 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.028133 14314 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.103917 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.109218 14806 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:01.110680 14783 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.111462 11516 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
W20260709 13:16:01.111958 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:01.073953 14802 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:16:01.113685 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 827) took 1592 ms. Trace:
I20260709 13:16:01.014816 14796 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.114040 12067 rpcz_store.cc:276] 0709 13:15:59.521153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.587069 (+ 65916us) service_pool.cc:224] Handling call
0709 13:16:00.040361 (+453292us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:16:00.041270 (+   909us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:00.432059 (+390789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:00.432093 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:00.450055 (+ 17962us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.900507 (+450452us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:00.900699 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.901748 (+  1049us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.113326 (+211578us) raft_consensus.cc:1685] finished
0709 13:16:01.113345 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.113530 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42915,"raft.queue_time_us":46751,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":29565312,"thread_start_us":784758,"threads_started":2,"wal-append.queue_time_us":588211}
I20260709 13:16:01.115548 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.115762 14241 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:16:01.116508 14736 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.602s	user 0.005s	sys 0.005s
I20260709 13:16:01.100047 14816 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:01.119311 14816 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:01.119226 14736 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.120885 14609 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:01.121544 14609 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:01.122097 14736 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.122742 14608 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:01.123819 14806 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:01.125696 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.125912 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.128269 14635 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.129284 11438 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:01.129616 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:01.130551 14635 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.131587 14791 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:01.132336 14791 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:01.132699 14791 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:01.133052 14791 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:01.135118 14804 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:01.136446 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 639) took 1318 ms. Trace:
I20260709 13:16:01.136806 12066 rpcz_store.cc:276] 0709 13:15:59.818477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.098802 (+280325us) service_pool.cc:224] Handling call
0709 13:16:00.642526 (+543724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:00.649801 (+  7275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.136268 (+486467us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b5400751050. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:01.136389 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247325440}
I20260709 13:16:01.131263 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.139379 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48186 (request call id 0) took 1148 ms. Trace:
I20260709 13:16:01.143676 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.159456 12063 rpcz_store.cc:276] 0709 13:15:59.996052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.026511 (+ 30459us) service_pool.cc:224] Handling call
0709 13:16:01.139174 (+1112663us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b5400522bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f8853 00007f35c19c660a 00007f35baf4c7d4 00007f35baf4c762 00007f35baf4c691 00007f35baf4c60c 00007f35baf4b8c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:01.139284 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930388864}
I20260709 13:16:01.173683 11366 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:01.174675 11444 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:16:01.188627 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45038.3us
I20260709 13:16:01.190831 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.147886 14776 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.194437 11363 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:01.196309 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.198035 11362 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:01.131265 14806 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:01.140365 14636 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:01.197075 11366 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
W20260709 13:16:01.206650 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 265651us
I20260709 13:16:01.207893 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.209573 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:01.212062 11366 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:01.188393 14358 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:01.188645 14737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.000s	sys 0.020s
I20260709 13:16:01.217810 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:01.234373 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 162009us
I20260709 13:16:01.234995 11363 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:01.243083 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.243885 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.244159 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.244606 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:01.244979 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 225268us
I20260709 13:16:01.244684 12066 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:01.245844 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.207271 14806 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:01.245362 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.247175 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.248121 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.250428 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.250655 11366 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:01.218616 14635 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.253168 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.255662 11363 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:01.255856 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.258499 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:01.259102 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.259541 12069 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.255700 12033 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:01.256352 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.260352 12063 raft_consensus.cc:2410] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:01.261145 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.263521 12067 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:01.265681 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.266918 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.243800 14247 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:01.269675 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 324041us
I20260709 13:16:01.270411 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.271126 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:01.272753 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.273350 12060 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:01.246373 14213 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.242713 14690 consensus_queue.cc:1243] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:01.260838 14382 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:01.279526 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.250240 14630 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:01.280100 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.285962 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.278820 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.280668 12060 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:16:01.262547 14736 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:01.282085 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 640) took 1463 ms. Trace:
I20260709 13:16:01.289445 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:01.271714 14700 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:01.277484 14783 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:01.275064 14635 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.320606 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.331673 11335 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.345501 11335 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.340749 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2021) took 1451 ms. Trace:
I20260709 13:16:01.329680 12066 raft_consensus.cc:2410] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:16:01.343276 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:01.349211 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 83506.1us
W20260709 13:16:01.344581 12064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 828) took 1744 ms (client timeout 2137 ms). Trace:
W20260709 13:16:01.349762 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 63897.1us
W20260709 13:16:01.354712 12064 rpcz_store.cc:269] 0709 13:15:59.584547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.584661 (+   114us) service_pool.cc:224] Handling call
0709 13:16:01.328326 (+1743665us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54004a6fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:01.328436 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1756148992}
W20260709 13:16:01.344584 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 103529us
I20260709 13:16:01.292639 14776 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:01.358068 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.292683 14736 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:01.297821 14352 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:01.298826 14609 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:01.294600 12061 rpcz_store.cc:276] 0709 13:15:59.818743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.101385 (+282642us) service_pool.cc:224] Handling call
0709 13:16:00.101731 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:00.102111 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:00.435044 (+332933us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.789867 (+354823us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:00.867892 (+ 78025us) log.cc:844] Serialized 52 byte log entry
0709 13:16:00.868041 (+   149us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:00.868272 (+   231us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.143049 (+274777us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b1800420d60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf 00007f35bca18038
0709 13:16:01.143288 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.277809 (+134521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.281850 (+  4041us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.281894 (+    44us) raft_consensus.cc:1685] finished
0709 13:16:01.281906 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.282032 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52803,"prepare.queue_time_us":338011,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":1987,"raft.queue_time_us":55929,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":65812,"spinlock_wait_cycles":4922752,"thread_start_us":791037,"threads_started":3,"wal-append.queue_time_us":562572}
I20260709 13:16:01.328331 14636 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.364037 11519 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:01.330657 14783 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:01.303562 14811 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:01.387367 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.365422 11334 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.366031 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.282452 14213 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.297545 14700 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.396641 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:01.369498 12069 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:01.356683 14241 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:01.303701 14247 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:16:01.300065 14816 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.352934 14358 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.392251 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.415812 11513 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:16:01.349468 11517 rpcz_store.cc:276] 0709 13:15:59.889317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.889618 (+   301us) service_pool.cc:224] Handling call
0709 13:15:59.909813 (+ 20195us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:00.283648 (+373835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:00.283680 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:16:00.284347 (+   667us) log.cc:844] Serialized 78 byte log entry
0709 13:16:00.678177 (+393830us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:01.031450 (+353273us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.298611 (+267161us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.340386 (+ 41775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.340463 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.340504 (+    41us) raft_consensus.cc:1685] finished
0709 13:16:01.340517 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.340678 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4454,"prepare.queue_time_us":67515,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":66,"raft.queue_time_us":35929,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":6802,"thread_start_us":1304283,"threads_started":4,"wal-append.queue_time_us":673183}
W20260709 13:16:01.393235 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.395534 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:01.417508 11515 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:01.391646 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.389895 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:01.395293 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.400444 12068 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:16:01.402588 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:01.404987 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.407666 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:01.366787 14836 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.405975 14630 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:01.360816 14806 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.410380 14213 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.410977 14816 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.417853 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.419253 11514 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
W20260709 13:16:01.424193 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 153151us
I20260709 13:16:01.424973 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:01.395221 14783 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.399353 14700 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:01.435124 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 69978.5us
I20260709 13:16:01.435892 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:01.437328 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:01.429561 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 82759.7us
I20260709 13:16:01.391366 14811 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:01.391862 14824 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:01.416006 14820 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:16:01.441812 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 51999.5us
I20260709 13:16:01.448117 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.366987 14609 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:01.445058 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:01.348889 14791 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.430451 14770 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.367678 14695 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:01.368409 14736 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.403107 14827 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:16:01.431257 14241 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:01.484215 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:01.484640 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48890.9us
I20260709 13:16:01.500036 11514 raft_consensus.cc:2368] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
W20260709 13:16:01.485509 14830 log.cc:927] Time spent T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:16:01.486109 14635 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.487979 14791 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:01.489611 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:01.512989 11370 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:01.490195 12064 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:01.476001 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:01.513726 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:01.435328 14842 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:01.490626 11445 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:01.497174 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 134605us
I20260709 13:16:01.446272 14811 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:01.454447 14609 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.447556 14833 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:01.475088 14816 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:01.507179 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:01.514915 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 73765.7us
I20260709 13:16:01.519502 11515 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:16:01.523540 14247 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:01.524281 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.427806 14836 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.527920 11514 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 3.
I20260709 13:16:01.526574 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:01.537732 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.543895 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:01.545447 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.546213 11445 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:01.547377 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:01.552963 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:01.553194 14852 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:01.555457 14834 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:01.555601 14795 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:01.556121 14795 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.534392 14824 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:01.560240 14824 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.560865 14852 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.553387 14791 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:16:01.562016 14791 consensus_queue.cc:1243] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:16:01.564604 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 62267.9us
I20260709 13:16:01.565297 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.526979 14833 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:01.566618 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.553325 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.564354 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.554747 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.537890 14846 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:16:01.577207 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70115.6us
I20260709 13:16:01.592383 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.572343 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.571046 11445 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:01.576292 14816 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:01.553443 14825 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:01.577345 14736 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.578081 11436 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:01.534452 14835 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:01.587524 14796 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.525848 14382 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:16:01.535370 14776 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:16:01.604491 11515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:16:01.606710 11515 raft_consensus.cc:606] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.607306 11515 raft_consensus.cc:614] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:01.621656 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.622537 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.561352 14852 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.625617 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.538882 14829 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:16:01.572088 14833 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.629391 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.635180 11511 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:16:01.597339 14791 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.639385 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.639845 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.640143 12069 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:16:01.604784 14825 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.641372 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.649299 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:01.649449 12069 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.643561 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.644572 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:01.606413 14827 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:01.642102 14849 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:01.639247 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 43697.2us
I20260709 13:16:01.643028 11510 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:01.605685 14842 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:01.652094 11513 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:01.607141 14835 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:01.658784 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:01.659565 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.662401 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.626600 14314 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.637823 14816 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:01.638594 14635 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:01.671736 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.673460 14736 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.626654 14241 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.675206 14247 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:01.662468 14849 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.678277 14638 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.679840 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 615) took 1004 ms. Trace:
I20260709 13:16:01.653929 14825 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:01.670717 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 78405.7us
I20260709 13:16:01.670646 14858 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.597579 14856 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:16:01.654561 14827 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.605892 14601 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:16:01.718303 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.723552 11515 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:01.660689 14835 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.727025 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.712529 11442 rpcz_store.cc:276] 0709 13:16:00.675005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.675226 (+   221us) service_pool.cc:224] Handling call
0709 13:16:00.705527 (+ 30301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:00.705784 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:01.035438 (+329654us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.289071 (+253633us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.673576 (+384505us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b18003e3b60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1
0709 13:16:01.674108 (+   532us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18003e3b60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:01.678976 (+  4868us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b18003e3b60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf 00007f35bca18038
0709 13:16:01.679220 (+   244us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.679392 (+   172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:01.679500 (+   108us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:01.679544 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.679609 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.679756 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23565,"prepare.queue_time_us":684383,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":115,"raft.queue_time_us":14251,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":35375,"spinlock_wait_cycles":209936000,"thread_start_us":579141,"threads_started":2,"wal-append.queue_time_us":399610}
I20260709 13:16:01.655115 14791 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.737075 12069 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:01.714871 14247 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:01.739878 11519 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:01.742074 12065 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
W20260709 13:16:01.745008 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 194762us
W20260709 13:16:01.666041 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 93779.1us
I20260709 13:16:01.746997 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.722878 14695 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:01.750151 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.750612 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.726850 14827 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.752127 12066 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:01.750586 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.754069 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.755270 11519 raft_consensus.cc:1217] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 3.6->[5.7-8.9]   Dedup: 5.7->[6.8-8.9]
I20260709 13:16:01.689894 14314 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.756076 11519 pending_rounds.cc:85] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 7
I20260709 13:16:01.763070 11519 pending_rounds.cc:107] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:16:01.763494 11519 raft_consensus.cc:2889] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:01.751302 14770 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:01.756003 14736 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.767491 14816 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:01.774637 14791 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:01.759363 14630 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:01.791208 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:01.794363 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 111225us
W20260709 13:16:01.797178 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 81897us
I20260709 13:16:01.802809 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.803460 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.807682 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.808482 12060 raft_consensus.cc:2368] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:16:01.816871 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.817668 11363 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:01.818511 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.819379 12063 raft_consensus.cc:2410] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:01.819737 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.820497 12061 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:01.821484 14833 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.823659 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:01.820576 14859 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:16:01.824118 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:01.795385 14864 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.799691 14849 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:01.825350 12069 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:01.727507 14858 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.831177 11366 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.832422 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.832655 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:01.833449 11369 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:01.837709 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:01.838049 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.843165 11362 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:01.823503 14816 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:01.846558 11364 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:01.835283 14770 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:01.851114 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:01.847996 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.852438 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.856006 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.856853 14700 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.862030 12067 raft_consensus.cc:2368] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
W20260709 13:16:01.863884 14787 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.347s	user 0.003s	sys 0.003s
I20260709 13:16:01.857712 14695 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:01.839951 14601 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:01.858682 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.869292 14827 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.872334 14736 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:01.843963 14849 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:01.873185 14849 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.845814 14864 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:16:01.826496 14829 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:16:01.875265 14314 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:01.866060 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 740) took 1889 ms. Trace:
I20260709 13:16:01.876235 14611 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:01.860206 14816 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.876617 14611 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:01.877480 14611 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.878604 14611 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.876117 11368 rpcz_store.cc:276] 0709 13:15:59.976360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.977446 (+  1086us) service_pool.cc:224] Handling call
0709 13:15:59.977751 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:59.978092 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:59.978118 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:59.978568 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:16:00.272189 (+293621us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:00.561042 (+288853us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.563931 (+  2889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.564116 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.830839 (+1266723us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:01.865743 (+ 34904us) raft_consensus.cc:1685] finished
0709 13:16:01.865858 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.865999 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":402061,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":6678,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":32252,"spinlock_wait_cycles":71617280,"thread_start_us":560042,"threads_started":2,"wal-append.queue_time_us":513066}
I20260709 13:16:01.884596 14700 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:01.885489 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:01.886027 14864 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:01.887895 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.888165 12062 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:01.889063 14247 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.890262 12069 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:16:01.890986 14247 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:01.850836 14863 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:01.891835 14863 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.892175 14863 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:01.895776 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:01.876083 14870 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:01.896363 11442 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:01.896436 14870 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.896895 14870 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:01.897536 11366 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:01.895712 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.899931 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:01.900493 11444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 610) took 1786 ms (client timeout 2078 ms). Trace:
W20260709 13:16:01.900876 11444 rpcz_store.cc:269] 0709 13:16:00.113715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.114474 (+   759us) service_pool.cc:224] Handling call
0709 13:16:00.781349 (+666875us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:16:01.900331 (+1118982us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54003e58d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:01.900434 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909367040}
I20260709 13:16:01.901888 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.902101 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:01.902667 14630 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:01.903342 14630 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:01.903697 14630 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:01.872502 14836 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.904448 14866 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.906052 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.910216 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.911826 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.903752 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:01.846421 14609 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.915484 14609 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.904116 14630 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.872231 14852 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:01.917976 14852 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:01.920595 14690 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:01.921062 14690 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:01.921875 14690 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.922860 14690 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:01.915614 14738 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:01.924155 14738 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:01.924465 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:01.915977 14837 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.419s	user 0.003s	sys 0.000s
I20260709 13:16:01.929028 14736 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:01.929533 14736 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:01.930388 14736 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.930802 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:01.930605 14825 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:01.911000 14827 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:01.945713 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 51739.1us
I20260709 13:16:01.949142 11441 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:01.952098 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.953943 14213 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:01.954088 14795 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:01.954605 14213 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.954988 14858 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:01.956244 14795 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:01.957168 11483 leader_election.cc:336] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.590s (SENT)
I20260709 13:16:01.957381 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:01.967105 12061 raft_consensus.cc:2410] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:16:01.968160 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 112218us
I20260709 13:16:01.969205 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:01.976872 12064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:01.977437 12064 raft_consensus.cc:606] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:01.950508 14806 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:01.982052 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:01.979588 11362 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:01.985061 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.988677 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:01.989319 11442 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:01.989349 12064 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:01.909590 14783 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:01.991896 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:01.992252 11369 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:01.970669 14784 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:01.994987 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:01.995378 11370 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:01.999433 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:02.000073 11362 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:01.979620 14858 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:02.001323 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.995575 11436 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:02.003770 11363 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:02.004861 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:02.004882 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:02.005861 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:02.002096 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.008608 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.009268 11436 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:01.976575 14738 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:16:01.998476 14864 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:02.014643 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:02.015275 12060 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:02.015664 11334 leader_election.cc:400] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:02.015998 11334 leader_election.cc:403] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:16:02.009876 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:02.010095 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:02.010352 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 68722.3us
I20260709 13:16:02.017275 11409 leader_election.cc:400] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:16:01.958249 14870 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.012006 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:01.993858 14849 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:02.015071 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:01.970605 14811 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.019974 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:02.019466 11409 leader_election.cc:403] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:16:01.975304 14835 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.019224 14870 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.021062 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:02.021062 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.023491 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.024446 12067 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:02.022370 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.026459 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.026635 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.027031 12069 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.027999 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.027009 11363 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.031790 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.032320 11439 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:16:02.029341 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.019742 14635 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:16:02.037667 14783 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:02.038841 14833 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.040297 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:02.003547 14630 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:02.043685 14630 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:02.022349 14700 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:16:02.047636 14855 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:02.049500 14835 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.050560 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:02.051941 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.053114 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.053829 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:02.054147 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 704) took 1504 ms. Trace:
I20260709 13:16:02.054865 14247 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.055459 14247 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.056563 11437 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:02.057735 14247 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:02.059469 12067 raft_consensus.cc:2410] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:16:02.059918 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:02.060380 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.060540 12062 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:16:02.060957 12067 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.065126 14784 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.072405 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:02.073161 11513 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:02.075263 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:02.054294 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.080523 12060 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:02.054570 11516 rpcz_store.cc:276] 0709 13:16:00.550082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.550216 (+   134us) service_pool.cc:224] Handling call
0709 13:16:00.556305 (+  6089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:01.111894 (+555589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.053912 (+942018us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540031f454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:02.054088 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276204288}
I20260709 13:16:01.997706 14863 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.067265 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:02.095010 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 77859.5us
I20260709 13:16:02.075927 11513 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
W20260709 13:16:02.092792 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 52642.6us
I20260709 13:16:02.097539 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:02.101090 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:02.101780 11513 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.104458 11511 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:02.105077 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.104589 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.106247 11369 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:02.099792 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:02.108729 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:02.109453 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:02.107771 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2161) took 1283 ms. Trace:
I20260709 13:16:02.111444 11443 rpcz_store.cc:276] 0709 13:16:00.824233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.039522 (+215289us) service_pool.cc:224] Handling call
0709 13:16:01.053230 (+ 13708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:01.053426 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:01.064990 (+ 11564us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.956417 (+891427us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b1800369880. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e23a 00007f35bca08a2a
0709 13:16:02.104064 (+147647us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:02.104120 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:02.104168 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.107597 (+  3429us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:02.107713 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14913,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":9660,"raft.queue_time_us":125,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":11398912,"thread_start_us":147054,"threads_started":1,"wal-append.queue_time_us":1034170}
I20260709 13:16:02.113458 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:02.114008 11443 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.065714 14784 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.123569 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.123800 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.125285 14630 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:02.124255 11439 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:02.129195 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.129772 11362 raft_consensus.cc:2368] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9.
I20260709 13:16:02.130647 11483 leader_election.cc:400] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9.
I20260709 13:16:02.131024 11483 leader_election.cc:403] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:16:02.094385 14866 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:02.127328 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.132828 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.091846 14816 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:02.136422 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.109333 14783 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:02.105619 14775 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:16:02.138103 11515 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:02.177709 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.162146 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.122279 14827 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:02.169953 12069 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:02.180552 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.105482 14795 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:02.138139 11511 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:02.141620 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:02.179318 11439 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:02.139590 12067 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:02.196094 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:02.219738 11439 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:02.104912 14796 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:16:02.105340 14638 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:16:02.221397 11443 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
W20260709 13:16:02.222260 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 80703.6us
I20260709 13:16:02.222857 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.231115 11365 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:02.119863 14589 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:16:02.139580 14799 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:02.094352 14700 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:16:02.229642 11439 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:16:02.243692 11439 pending_rounds.cc:85] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 4
I20260709 13:16:02.243942 11439 pending_rounds.cc:107] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:16:02.244161 11439 raft_consensus.cc:2889] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:02.163318 14783 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.226372 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:02.195195 14795 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:02.139343 14630 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.230960 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:02.235023 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:02.120102 14314 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 13:16:02.220649 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:02.195583 14835 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:02.137475 14858 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:02.262228 14858 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.252140 14783 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.246949 11439 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:16:02.247258 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:02.266618 14784 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.220279 14825 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:02.255815 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.248464 11517 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
I20260709 13:16:02.273160 11361 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.259874 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:02.260597 11443 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:02.257822 14795 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:02.274569 11363 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:02.242648 14799 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:02.282678 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.281174 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.298622 11365 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.271535 14605 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:02.310575 12032 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:02.301816 11517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:40756
W20260709 13:16:02.316542 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:02.318285 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.276913 14652 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:02.323913 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.327491 11437 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:16:02.327894 11437 pending_rounds.cc:85] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 7
I20260709 13:16:02.328116 11437 pending_rounds.cc:107] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:16:02.328351 11437 raft_consensus.cc:2889] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:02.276423 14813 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:02.297706 14799 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.340500 14868 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:02.299023 14835 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.301136 14804 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 13:16:02.342597 14873 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:16:02.342319 14835 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:02.345057 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:02.339895 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 609) took 2223 ms. Trace:
I20260709 13:16:02.332737 14605 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:02.326586 14839 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:02.321218 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:02.352746 11511 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:02.345790 11437 rpcz_store.cc:276] 0709 13:16:00.112636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.112841 (+   205us) service_pool.cc:224] Handling call
0709 13:16:00.731027 (+618186us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:16:00.731278 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:02.056141 (+1324863us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54000b93a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc973a1d 00007f35bc95a1a4 00007f35bc9dc447 00007f35bc9f4fe7 00007f35bc9f67de 00007f35bca00b49 00007f35bca01416 00007f35bc9fdd8f 00007f35bc9fd83e 00007f35c19c2419
0709 13:16:02.335369 (+279228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.335665 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112704128}
W20260709 13:16:02.379266 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 188540us
I20260709 13:16:02.300470 14827 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:02.333701 14314 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:02.322090 14652 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:02.343559 14878 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:16:02.340438 14852 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:02.380796 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 59665.3us
I20260709 13:16:02.342171 14868 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:02.339195 14611 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:16:02.396559 12032 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:02.381330 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:02.337538 14846 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:16:02.380776 14875 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
W20260709 13:16:02.408926 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.384187 14314 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.341894 14871 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:16:02.384361 14791 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:02.420024 14633 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:02.420785 12060 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:02.421168 14886 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:16:02.423700 14851 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:02.424190 14851 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.424535 14851 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:02.423147 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:02.426959 12060 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:02.397229 14863 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.401214 14839 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:16:02.428999 14605 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.430052 11513 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:02.432886 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2367) took 1105 ms. Trace:
I20260709 13:16:02.433276 11441 rpcz_store.cc:276] 0709 13:16:01.327358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.741783 (+414425us) service_pool.cc:224] Handling call
0709 13:16:01.746573 (+  4790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:01.949616 (+203043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.432671 (+483055us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400398854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:02.432835 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465393920}
I20260709 13:16:02.413599 11518 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:16:02.419130 14314 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:02.419636 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 1.16156e+06us
I20260709 13:16:02.436677 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:02.398322 14827 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:02.444167 11436 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:02.404520 14352 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:02.441704 14633 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:02.456409 14633 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.419937 14868 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.400429 14848 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 13:16:02.465205 11361 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:02.407884 14864 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:02.462894 11365 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:02.414443 14838 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:02.470472 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:02.474591 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:02.475680 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 227216us
I20260709 13:16:02.488794 12067 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:16:02.436712 14852 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.493260 12060 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:02.463444 14352 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:02.497327 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.497874 11361 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:02.468776 14864 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.498826 12060 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:16:02.500555 12060 pending_rounds.cc:85] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 6
I20260709 13:16:02.500787 12060 pending_rounds.cc:107] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:16:02.501011 12060 raft_consensus.cc:2889] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:02.495028 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:02.499819 12033 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:02.488015 14609 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:02.510833 11365 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:02.493547 14638 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.498099 14835 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:02.518435 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:02.518842 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:02.520066 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:02.452126 14652 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.496259 14852 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.528126 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:02.540652 11512 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:02.527566 14638 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.538375 11363 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:02.542661 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:02.496917 14883 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 13:16:02.520411 11408 connection.cc:256] RPC call timeout handler was delayed by -1.1807335160001458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 13:16:02.515547 14589 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:02.546722 11519 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:02.553485 11361 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:02.516299 14609 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:02.557250 11408 connection.cc:256] RPC call timeout handler was delayed by -1.0556335160001709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
I20260709 13:16:02.561198 12067 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:02.514779 14827 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.571957 11519 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:02.560086 14652 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:02.581281 14893 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:16:02.581012 14609 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:02.563277 14589 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:02.596324 11512 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:02.603279 14611 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:02.606333 14611 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:02.605099 14904 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:16:02.544879 14352 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.619930 14352 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:02.651271 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47103.3us
I20260709 13:16:02.581583 14799 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.672032 14589 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.673671 14795 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:02.676316 12068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 866) took 1428 ms (client timeout 1757 ms). Trace:
W20260709 13:16:02.676720 12068 rpcz_store.cc:269] 0709 13:16:01.247821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.247925 (+   104us) service_pool.cc:224] Handling call
0709 13:16:02.676121 (+1428196us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b540039fdd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:02.676256 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344079744}
I20260709 13:16:02.677873 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.679148 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:02.680450 14690 consensus_meta.cc:287] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.757s	user 0.003s	sys 0.011s
I20260709 13:16:02.681900 12069 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:02.683355 12065 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:02.684123 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:02.687503 11335 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:02.653874 14887 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.335s	user 0.004s	sys 0.000s
I20260709 13:16:02.706197 11442 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
W20260709 13:16:02.711592 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.646723 14864 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.688757 14835 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:02.688539 14863 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:02.708159 14894 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.692222 14833 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:16:02.701370 14816 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:16:02.698091 14783 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.727587 14605 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:02.729548 14638 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:02.720204 14314 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.752049 14816 consensus_queue.cc:1243] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:02.765002 11517 raft_consensus.cc:606] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:02.798435 11517 raft_consensus.cc:614] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:02.799371 11518 raft_consensus.cc:2368] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:02.727655 14908 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:16:02.751340 14870 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:02.798014 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.755434 14611 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.806762 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:02.807499 11366 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:02.804787 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:02.800926 11409 leader_election.cc:336] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.500s (SENT)
I20260709 13:16:02.731876 14894 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.810698 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:02.819566 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:02.822983 11443 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:02.751122 14884 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:16:02.824512 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:02.826295 11445 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:02.808017 14903 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:16:02.794518 14852 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.735606 14851 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.833199 14910 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:02.834188 14916 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:16:02.840224 11445 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:02.846011 11366 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:02.760432 14605 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:02.850025 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:02.851559 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50671.9us
I20260709 13:16:02.842921 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:02.847304 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 96796.3us
I20260709 13:16:02.854830 14863 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:02.855121 14358 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:02.855562 14358 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:02.855942 14864 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:02.857430 14915 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:16:02.860167 14912 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:16:02.854605 14609 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:02.862111 14685 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:02.862601 14685 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:02.865501 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 655) took 1061 ms. Trace:
I20260709 13:16:02.865947 12065 rpcz_store.cc:276] 0709 13:16:01.804014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.804414 (+   400us) service_pool.cc:224] Handling call
0709 13:16:02.683016 (+878602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:02.683783 (+   767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.865275 (+181492us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b540039fdd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:02.865441 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1951483264}
I20260709 13:16:02.813190 14870 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:02.888427 11443 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:02.848309 14926 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:16:02.854494 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:02.897439 11437 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:16:02.897320 11436 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:16:02.905280 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52172.4us
I20260709 13:16:02.885617 14871 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:02.846819 14816 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:16:02.871354 14799 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:02.916105 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 61696.1us
I20260709 13:16:02.917811 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:02.920886 11436 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:16:02.918689 14914 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:16:02.914535 14816 consensus_queue.cc:1243] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:02.912312 14852 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:02.994973 14894 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:02.945823 14783 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:02.957981 14919 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:16:02.917599 14871 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:02.998070 14871 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:02.978128 14918 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:16:02.946388 14923 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:02.980111 14630 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.003s
I20260709 13:16:03.003019 14630 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9.
I20260709 13:16:02.983248 14314 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:02.980696 14928 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:16:02.981925 14870 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:02.984295 14605 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:02.988690 14635 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:02.989367 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.006726 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.007998 14783 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:03.008492 14783 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:03.009377 14783 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:02.989383 14601 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:16:02.991544 14925 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:02.993026 14851 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:03.014077 14630 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:03.014766 14630 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:03.015664 14630 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:03.019054 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 101306us
I20260709 13:16:03.031093 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:03.038379 14882 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.934s	user 0.001s	sys 0.001s
I20260709 13:16:03.038445 11514 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:03.044394 14796 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:03.044945 14796 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:03.045657 14863 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:03.045724 14796 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.046710 14863 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.047120 14796 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:03.039212 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:03.047508 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 153914us
I20260709 13:16:03.050155 14827 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:03.047279 11514 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:03.054944 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:03.057475 14824 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:03.062580 14769 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:03.063069 14769 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:03.063378 14813 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:03.063885 14769 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.064944 14769 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:03.067795 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.068450 11444 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:03.070755 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.062623 14824 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
W20260709 13:16:03.073559 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:03.074342 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.075248 14849 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.075810 14849 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.076754 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.077919 14849 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:03.078447 14849 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.078361 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.079437 11440 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:16:03.102562 11370 pending_rounds.cc:85] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 3
I20260709 13:16:03.100701 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:03.095702 11408 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:03.110119 14605 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:03.094287 14241 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:03.110999 14685 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:03.112804 14685 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.113317 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.113929 14906 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:03.114334 14906 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:03.115074 14906 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.101790 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:03.116024 14906 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:03.083222 14352 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:03.117594 14352 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:03.156853 14352 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:03.159350 14920 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.346s	user 0.008s	sys 0.001s
I20260709 13:16:03.095316 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.120275 14241 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:03.163722 14894 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:03.086476 14775 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:03.164116 14894 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.164417 14775 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:03.164585 14894 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.165263 14775 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:03.123703 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:03.167689 14923 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:03.108604 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.124924 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:03.169453 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44602.6us
I20260709 13:16:03.169772 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.170866 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.164079 14241 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:03.174054 14241 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.174381 14816 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:03.177862 14881 log.cc:927] Time spent T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.075s	user 0.006s	sys 0.000s
I20260709 13:16:03.124898 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.127938 11441 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:03.180459 11441 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:03.182615 14864 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:03.113932 11444 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:03.185647 11440 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:03.128748 11439 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:03.187987 14806 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:16:03.187913 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 91986.9us
W20260709 13:16:03.131645 14909 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.415s	user 0.003s	sys 0.000s
I20260709 13:16:03.125643 14811 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:03.135680 14835 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:16:03.137918 14922 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.310s	user 0.000s	sys 0.004s
I20260709 13:16:03.137856 14358 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.143702 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2167) took 1200 ms. Trace:
I20260709 13:16:03.143994 11443 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:03.231638 11442 rpcz_store.cc:276] 0709 13:16:01.943217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.996499 (+ 53282us) service_pool.cc:224] Handling call
0709 13:16:03.143488 (+1146989us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b5400399254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:03.143618 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553084672}
W20260709 13:16:03.150609 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.154963 14795 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:03.159830 14934 log.cc:927] Time spent T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.115s	user 0.004s	sys 0.000s
I20260709 13:16:03.080509 11511 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:03.168417 14923 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:03.235513 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:03.236232 11362 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:03.237071 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:03.185913 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.237751 11368 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:03.108597 11370 pending_rounds.cc:107] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:16:03.238395 11370 raft_consensus.cc:2889] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:03.186693 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:03.241380 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54768.4us
I20260709 13:16:03.241622 14899 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:03.242234 14899 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.242558 14899 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.244498 14905 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:03.078949 14849 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.189146 14806 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:03.247025 14806 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:03.189076 11439 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:16:03.248045 14849 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:03.247869 14806 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.249199 14806 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:03.249732 14899 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.191298 14833 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:03.251962 14899 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:03.252928 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:03.111445 11408 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:03.199663 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 104431us
I20260709 13:16:03.256727 14849 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:03.257315 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.258119 14589 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:03.258641 14589 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:03.258970 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.202601 14894 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.205781 14352 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.226549 14835 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:03.259570 14589 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.260217 14835 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:03.260964 14352 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:03.261595 11365 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:03.262710 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:03.193295 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.264427 11437 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:03.189056 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:03.265321 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:03.265918 11517 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:03.209671 14917 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
I20260709 13:16:03.213791 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.215133 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:03.270684 11510 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:03.216166 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.225366 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:40756
I20260709 13:16:03.271687 11516 raft_consensus.cc:606] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.272058 11516 raft_consensus.cc:614] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:03.228852 11519 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:03.270076 11518 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:03.271296 11514 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:03.253012 12033 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:03.255112 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1706) took 1061 ms. Trace:
I20260709 13:16:03.271701 14855 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:03.279207 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.280633 11439 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.260897 14894 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:03.212944 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:03.265360 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 76393.3us
W20260709 13:16:03.284011 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.279248 11369 rpcz_store.cc:276] 0709 13:16:02.193350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.193497 (+   147us) service_pool.cc:224] Handling call
0709 13:16:02.222530 (+ 29033us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:02.262476 (+ 39946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:02.262504 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:02.671880 (+409376us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b2c000c7fc8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:03.024840 (+352960us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b2c000c7fc8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:03.025426 (+   586us) log.cc:844] Serialized 52 byte log entry
0709 13:16:03.210641 (+185215us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:03.210682 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.210752 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:03.254887 (+ 44135us) raft_consensus.cc:1685] finished
0709 13:16:03.254908 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:03.255053 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12271,"raft.queue_time_us":25523,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"spinlock_wait_cycles":52137984,"thread_start_us":184802,"threads_started":1,"wal-append.queue_time_us":183626}
W20260709 13:16:03.288143 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 75280us
W20260709 13:16:03.311713 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 200344us
I20260709 13:16:03.312526 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:03.324116 11408 leader_election.cc:400] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:16:03.272764 14652 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:03.274933 14923 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:03.276441 14795 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:03.333952 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 50077.6us
W20260709 13:16:03.334075 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 48929.8us
W20260709 13:16:03.360725 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:03.360591 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47018.9us
I20260709 13:16:03.286739 14835 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:03.364688 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 105775us
I20260709 13:16:03.365638 12034 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:03.316720 14926 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:03.350730 14894 raft_consensus.cc:2764] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:03.316707 14833 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:03.277675 14605 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:16:03.333577 14855 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:03.380287 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 96209.4us
I20260709 13:16:03.381884 11439 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:03.384773 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:03.334019 14783 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:16:03.395471 11518 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:03.359212 14652 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.362166 14858 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:16:03.365659 14795 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.405772 14845 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:03.398424 14805 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:16:03.384439 14948 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:03.432646 11514 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:03.434579 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 181627us
I20260709 13:16:03.401258 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.399176 14314 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:03.479741 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 76078.9us
I20260709 13:16:03.481768 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:03.491129 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:03.491788 12061 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
W20260709 13:16:03.494383 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 94222us
I20260709 13:16:03.459561 14948 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.500963 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:03.480504 14845 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:03.506038 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:03.510455 11408 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:03.513181 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:03.516530 12065 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:03.499735 14776 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:03.499995 14804 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:03.504747 14926 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:03.521659 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 156114us
I20260709 13:16:03.503835 14948 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:03.529682 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:03.533377 12068 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:03.533885 11519 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:03.534911 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:03.535831 12068 raft_consensus.cc:2410] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:03.511765 14605 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:03.537902 12060 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:03.538365 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:03.539208 12068 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:03.521582 14804 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.520184 14776 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:03.544715 12065 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:03.547175 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:03.547863 12062 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:03.558354 12066 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:16:03.561094 11334 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:03.546365 14926 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:03.583752 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.584849 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:03.585071 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.586028 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:03.586972 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.588512 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:03.538743 14805 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:16:03.553018 14776 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:03.555852 14942 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:16:03.633886 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:03.636405 12062 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.637806 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:03.542953 14605 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:03.634701 14945 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:16:03.639088 14926 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:03.549647 14804 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.493523 14855 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:03.623710 12068 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:03.625458 11370 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:03.626313 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.628688 12034 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:03.598812 11519 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:03.598089 14769 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:03.600538 11441 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:03.640400 14652 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:03.604388 11517 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:03.695734 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:03.634227 14858 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:16:03.663794 12033 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:03.572217 14845 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:03.698066 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 126845us
I20260709 13:16:03.658601 14795 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:03.701930 11439 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:16:03.701288 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 186047us
I20260709 13:16:03.668558 14638 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:03.722645 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:03.724223 14851 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:03.726300 14851 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:03.727432 14848 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:03.727821 14848 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:03.728746 14848 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.730917 14848 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:03.732379 14901 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:03.736565 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.740967 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:03.746304 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 82609.1us
I20260709 13:16:03.748641 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:03.750654 14795 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:03.700639 14679 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:16:03.699735 14953 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:16:03.755752 14799 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:16:03.758903 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 245789us
I20260709 13:16:03.775141 11408 leader_election.cc:400] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:03.777103 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.778405 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:03.779057 11519 raft_consensus.cc:2376] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8. Re-sending same reply.
I20260709 13:16:03.790689 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.764350 14811 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.808125 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:03.808796 12060 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:03.810277 11370 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:03.795709 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:03.814813 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 73921us
W20260709 13:16:03.822855 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.860872 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:03.861414 12063 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:03.861636 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.862905 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.892802 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.896212 11515 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:03.897874 11517 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:03.905028 12066 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:03.906667 12062 raft_consensus.cc:2410] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:16:03.933983 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.934314 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:03.787629 14795 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.124s
W20260709 13:16:03.944981 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 129437us
I20260709 13:16:03.951568 12060 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
W20260709 13:16:03.942879 14950 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
W20260709 13:16:03.968055 14967 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 13:16:04.039618 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 86725.8us
I20260709 13:16:03.787897 14899 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:03.957420 14957 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 13:16:03.942412 14921 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:03.865149 14811 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:03.957165 14927 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:16:04.112063 14835 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.736s	user 0.008s	sys 0.005s
I20260709 13:16:03.985711 14605 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:04.123206 14241 consensus_meta.cc:287] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.949s	user 0.012s	sys 0.007s
I20260709 13:16:04.112447 14679 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.112847 14314 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:04.179165 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:04.130839 14783 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.975234 14804 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.137439 14921 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:04.169371 14633 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:04.137189 14927 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:04.225359 14958 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.556s	user 0.005s	sys 0.001s
I20260709 13:16:04.225148 14927 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:04.215998 14960 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.442s	user 0.001s	sys 0.000s
I20260709 13:16:04.226389 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2191) took 1102 ms. Trace:
W20260709 13:16:04.217886 14965 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:16:04.218447 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.226730 11438 rpcz_store.cc:276] 0709 13:16:03.124192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.130894 (+  6702us) service_pool.cc:224] Handling call
0709 13:16:03.131202 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.135434 (+  4232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.135468 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.211472 (+ 76004us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:03.211990 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:16:03.506396 (+294406us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:03.506436 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.506502 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.224686 (+718184us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:04.226164 (+  1478us) raft_consensus.cc:1685] finished
0709 13:16:04.226186 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.226333 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3797,"raft.queue_time_us":4014,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":92211328,"thread_start_us":275391,"threads_started":1,"wal-append.queue_time_us":413546}
W20260709 13:16:04.218726 11268 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39438 (request call id 359) took 15773 ms (client timeout 15000 ms). Trace:
I20260709 13:16:04.223264 12064 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:04.133632 14899 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:04.233152 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2190) took 1111 ms. Trace:
W20260709 13:16:04.228487 11268 rpcz_store.cc:269] 0709 13:15:48.445147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.445734 (+   587us) service_pool.cc:224] Handling call
0709 13:16:04.218521 (+15772787us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:57.419684 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:57.669912 (+250228us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:57.669944 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:15:57.669961 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:58.084006 (+414045us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:58.084066 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:58.084097 (+    31us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:15:58.185065 (+100968us) tablet.cc:701] Row locks acquired
0709 13:15:58.186987 (+  1922us) write_op.cc:260] PREPARE: finished
0709 13:15:58.199847 (+ 12860us) write_op.cc:270] Start()
0709 13:15:58.199995 (+   148us) write_op.cc:276] Timestamp: P: 1783602958199779 usec, L: 0
0709 13:15:58.200021 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:15:58.201017 (+   996us) log.cc:844] Serialized 6516 byte log entry
0709 13:15:58.687923 (+486906us) op_driver.cc:464] REPLICATION: finished
0709 13:15:58.756565 (+ 68642us) write_op.cc:301] APPLY: starting
0709 13:15:58.756633 (+    68us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:58.756812 (+   179us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:58.756922 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:58.758138 (+  1216us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:58.758435 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:15:58.758473 (+    38us) write_op.cc:312] APPLY: finished
0709 13:15:58.758836 (+   363us) log.cc:844] Serialized 113 byte log entry
0709 13:15:58.758991 (+   155us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:58.828542 (+ 69551us) write_op.cc:454] Released schema lock
0709 13:15:58.828718 (+   176us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:58.829842 (+  1124us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:58.829888 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1052347,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":56921,"apply.run_cpu_time_us":3964,"apply.run_wall_time_us":73111,"num_ops":16,"prepare.queue_time_us":814341,"prepare.run_cpu_time_us":30435,"prepare.run_wall_time_us":897708,"raft.queue_time_us":379458,"raft.run_cpu_time_us":19419,"raft.run_wall_time_us":107679,"replication_time_us":466114,"thread_start_us":1459239,"threads_started":4,"wal-append.queue_time_us":56903,"wal-append.run_cpu_time_us":50848,"wal-append.run_wall_time_us":1575575}]]}
I20260709 13:16:04.235671 11438 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:04.206331 14921 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:04.233549 11445 rpcz_store.cc:276] 0709 13:16:03.122119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.122852 (+   733us) service_pool.cc:224] Handling call
0709 13:16:03.123140 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.123487 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.123515 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.388720 (+265205us) spinlock_profiling.cc:243] Waited 51.9 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:03.392126 (+  3406us) log.cc:844] Serialized 26 byte log entry
0709 13:16:03.542270 (+150144us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:03.542331 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.542451 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.232880 (+690429us) raft_consensus.cc:1685] finished
0709 13:16:04.232899 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.233098 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":348,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":77,"spinlock_wait_cycles":119472512,"thread_start_us":90242,"threads_started":1,"wal-append.queue_time_us":245400}
W20260709 13:16:04.251879 14969 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:16:04.253362 14973 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:16:04.233268 14848 consensus_meta.cc:287] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.502s	user 0.003s	sys 0.007s
I20260709 13:16:04.241005 14633 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.174437 14923 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 13:16:04.265626 14948 consensus_meta.cc:287] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.724s	user 0.008s	sys 0.000s
I20260709 13:16:04.242262 14899 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.193152 14679 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:04.269456 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.269893 12060 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:04.204416 14783 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.204027 14314 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.282574 14948 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:04.300499 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.300860 14848 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:04.312837 11513 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:04.326974 14952 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.327626 14952 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.329649 14952 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.267478 14901 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:16:04.298537 14899 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.335135 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.337746 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.340000 11445 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:04.339942 11515 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:04.356702 14948 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.350323 14783 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.350454 14979 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.363777 14923 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:04.349653 14899 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.413429 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.415339 14899 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:04.377955 14806 consensus_meta.cc:287] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.128s	user 0.011s	sys 0.001s
I20260709 13:16:04.416157 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.416831 11515 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:04.416668 14923 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.420192 14679 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.383925 14947 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:04.392316 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:04.422509 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:04.399699 11444 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:04.422919 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:04.422775 14947 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.403858 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.422508 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:04.407517 14811 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.424019 12068 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:04.424371 11408 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:04.408622 14804 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.409277 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:04.409694 14633 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.368969 14979 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.410326 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.459709 12068 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:04.459950 11511 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:04.425418 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:04.424019 11445 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:04.462524 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:04.423708 14827 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:04.412670 14314 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.464294 11408 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:04.467679 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.468389 11515 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:04.470971 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:04.471591 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:04.472246 11511 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:04.475833 14989 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:04.476811 14989 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:04.477129 14989 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:04.477836 14989 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.478938 14989 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:04.469308 14953 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:04.482033 11408 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:04.482385 14953 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:04.424401 14921 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.483870 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:04.484246 14827 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:04.485023 14827 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:04.485925 14863 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.486419 14863 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:04.462854 14988 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.487643 14988 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.488432 14863 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:04.489773 14988 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.492165 14949 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:04.460464 14783 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:04.461567 14795 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.463655 14633 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:04.498777 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:04.483225 14953 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:04.502689 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.483249 14927 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:04.500098 14824 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.515377 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.502588 14633 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:04.503746 14769 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:04.520973 12068 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:04.499758 14923 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:04.510988 14776 consensus_meta.cc:287] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.869s	user 0.009s	sys 0.003s
W20260709 13:16:04.529515 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.501960 14783 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:04.629173 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:04.503772 14795 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:04.643425 14955 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:16:04.645063 14979 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.646210 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:04.523291 14824 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.523736 14633 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:04.621493 14769 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.649003 14952 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:04.649802 14899 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:16:04.650509 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 187606us
I20260709 13:16:04.633562 14811 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:04.655014 14314 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:04.614712 14804 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.670168 14783 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:04.624853 14901 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:04.717563 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.718063 14901 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:04.718807 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.626367 14776 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:04.674242 14981 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.675560 14858 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.678988 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.679329 14685 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:04.682719 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.738692 11442 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:04.696725 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:04.700443 11437 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:04.737145 11444 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:04.740293 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.732892 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.648157 14921 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.653481 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:04.742177 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88792.8us
I20260709 13:16:04.708386 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.743994 11514 raft_consensus.cc:2410] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:16:04.709683 11518 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:04.745621 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:04.711445 11517 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:04.720144 14901 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.724071 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.741012 11437 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:04.728667 11511 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:04.730237 11408 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:04.751003 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:04.751255 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.734678 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.752934 11511 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:04.741246 11516 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:04.750265 11518 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:04.750244 11510 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:04.757534 11445 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:04.757573 11408 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:04.758165 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.758899 11514 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:04.757812 11517 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:04.714607 14952 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:04.754783 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:04.768164 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 93018.4us
I20260709 13:16:04.771080 11408 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:04.727916 14863 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:16:04.716423 14811 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:04.773865 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:04.727177 14804 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:04.777619 14996 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:16:04.772125 14314 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:04.767382 14979 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:04.775228 14827 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:04.748772 14858 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:04.762614 14901 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:04.821853 14901 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.739991 14981 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.825030 14981 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.785903 14952 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.786460 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2394) took 1238 ms. Trace:
I20260709 13:16:04.825735 11440 rpcz_store.cc:276] 0709 13:16:03.548340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.624060 (+ 75720us) service_pool.cc:224] Handling call
0709 13:16:03.624370 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.774207 (+149837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.774238 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.793058 (+ 18820us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.028822 (+235764us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:04.127572 (+ 98750us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.493164 (+365592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.786098 (+292934us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54003156d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:04.786144 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.786186 (+    42us) raft_consensus.cc:1685] finished
0709 13:16:04.786269 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.786405 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63122,"prepare.queue_time_us":101742,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":22523,"raft.queue_time_us":263521,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":4023,"spinlock_wait_cycles":23406464,"thread_start_us":554917,"threads_started":3,"wal-append.queue_time_us":277423}
I20260709 13:16:04.788028 14953 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:04.830888 14804 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.832572 14811 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:04.789351 14769 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:04.790277 15000 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:16:04.792500 14946 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.333s	user 0.005s	sys 0.003s
I20260709 13:16:04.773597 14921 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:16:04.773487 14923 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:16:04.797609 14795 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.798755 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:04.790174 14358 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:04.748251 14685 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:04.800699 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 86665.6us
I20260709 13:16:04.749020 14799 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:04.801512 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:04.757669 14848 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:04.761953 14995 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:04.764580 14849 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:16:04.815786 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:04.816905 11512 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:04.805217 14824 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.810631 14314 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:04.855373 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:04.859469 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 58051.2us
I20260709 13:16:04.877601 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:04.867151 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2186) took 1754 ms. Trace:
I20260709 13:16:04.870043 14685 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.871615 14858 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.872314 14901 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.873075 14776 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:04.867789 14795 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:04.882691 14685 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:04.886034 14986 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.526s	user 0.001s	sys 0.001s
I20260709 13:16:04.866663 14848 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.878852 14314 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:04.889848 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 74173.9us
I20260709 13:16:04.893159 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.905298 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.905670 11515 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:04.906584 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.907580 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.907851 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.908427 11514 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:04.905882 14776 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.909657 14776 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.911607 14776 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.913400 12063 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:04.914247 14799 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.914713 14799 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:04.916602 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.917516 11516 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:04.893548 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.921016 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:04.921497 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:04.922204 14925 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.905892 11510 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:04.879657 11436 rpcz_store.cc:276] 0709 13:16:03.112274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.114022 (+  1748us) service_pool.cc:224] Handling call
0709 13:16:03.114352 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:03.114726 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.114752 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:03.311533 (+196781us) spinlock_profiling.cc:243] Waited 60.8 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:03.312141 (+   608us) log.cc:844] Serialized 52 byte log entry
0709 13:16:03.507679 (+195538us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:03.528742 (+ 21063us) log.cc:844] Serialized 14 byte log entry
0709 13:16:03.704337 (+175595us) log.cc:844] Serialized 14 byte log entry
0709 13:16:03.740500 (+ 36163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.740579 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.487192 (+746613us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b180046c3c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:04.866934 (+379742us) raft_consensus.cc:1685] finished
0709 13:16:04.866955 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.867090 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100978,"prepare.queue_time_us":21799,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":271092,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":55703,"spinlock_wait_cycles":146325760,"thread_start_us":307867,"threads_started":2,"wal-append.queue_time_us":194576}
I20260709 13:16:04.852631 14633 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.898770 14795 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:04.909147 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.929193 11515 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:04.927048 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:04.933367 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:04.937474 15004 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:04.940344 15004 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:04.940735 15004 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:04.935005 14883 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:04.941533 14883 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:04.941867 14883 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:04.935284 14776 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:16:04.945039 14795 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.946179 14923 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:04.946539 14923 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:04.938330 14824 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:16:04.947294 14923 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.938490 14833 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:04.949074 14833 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:04.949440 14833 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:04.932360 14989 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:16:04.932500 14925 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.956466 14925 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.958019 14848 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.958407 14883 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:04.959816 14633 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:04.973454 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:04.974650 12034 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:04.939698 14358 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:04.992204 14358 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:04.992493 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.993626 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:04.994127 11513 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:04.994992 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:04.979938 14638 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:04.995772 11513 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:04.980247 14352 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:04.996184 14352 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:04.997932 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:05.014885 11512 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:05.021035 12034 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:05.000720 14833 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:05.002326 14883 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:05.006134 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:05.014901 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.015625 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:05.016983 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:05.037405 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.042968 12064 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:05.043804 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.019414 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.045343 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.046648 12060 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:05.048237 11518 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:05.049516 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:05.052269 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.057207 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:05.059331 12060 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:05.013361 15004 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.058312 14776 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:05.022332 14352 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:05.031708 14700 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:05.067997 11514 raft_consensus.cc:2368] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:05.072214 12069 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:05.022143 14358 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:05.077680 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:05.013270 14314 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.034235 14997 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.034132 14998 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:05.090006 14998 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.093309 14998 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:05.094246 14998 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:05.061126 14848 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.084851 11518 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:05.050699 14652 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:05.058430 14633 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.058790 15008 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:05.096978 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 673) took 1589 ms. Trace:
I20260709 13:16:05.061797 14589 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:05.016299 14952 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 13:16:05.134375 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.058903 14974 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:05.088352 14989 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:05.059156 14247 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:16:05.137523 15010 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:16:05.086980 14804 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:05.152160 12061 rpcz_store.cc:276] 0709 13:16:03.507256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.507620 (+   364us) service_pool.cc:224] Handling call
0709 13:16:03.507947 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.508325 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.508352 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.964961 (+1456609us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b2c000a6788. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:04.965365 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.049063 (+ 83698us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:05.078361 (+ 29298us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.080138 (+  1777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.080212 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.080247 (+    35us) raft_consensus.cc:1685] finished
0709 13:16:05.096789 (+ 16542us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.096925 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38313,"prepare.queue_time_us":26660,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2204,"raft.queue_time_us":3276,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":14729,"spinlock_wait_cycles":8000768,"thread_start_us":83358,"threads_started":1,"wal-append.queue_time_us":99874}
W20260709 13:16:05.162552 15015 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:05.164443 11518 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:05.158527 14589 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.199645 14589 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:05.150292 14633 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.201957 14633 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:05.088608 14776 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.205024 14776 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:05.160000 14990 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:16:05.205178 14358 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.207311 14358 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:05.138302 14848 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.098961 14690 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 13:16:05.160331 15005 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:16:05.210399 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:05.210536 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:05.099222 14997 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.211681 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.212324 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:05.203181 15004 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:05.153348 14863 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:05.152052 14974 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:05.212842 15018 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:05.224587 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:05.225255 11519 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:05.245885 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.254452 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:05.255541 12061 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:05.211051 15019 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:16:05.255128 12066 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:05.266373 14905 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:05.223673 14974 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:05.266997 14905 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:05.273145 14980 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.907s	user 0.010s	sys 0.007s
I20260709 13:16:05.273834 11364 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:05.195705 14804 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.393440 14899 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:05.245966 14638 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:05.228036 15018 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.273029 14352 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:05.156296 15008 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:05.509647 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:06.198262 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.521615 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:06.197492 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:05.952677 11519 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.727s	user 0.001s	sys 0.005s
I20260709 13:16:06.438766 11519 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:05.816526 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.440413 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.441286 11512 raft_consensus.cc:2410] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:16:06.027271 11441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.010s	sys 0.030s
W20260709 13:16:05.879621 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:05.924113 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 833311us
W20260709 13:16:06.368564 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.006s	sys 0.007s
W20260709 13:16:06.443562 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.007s	sys 0.009s
I20260709 13:16:06.448516 15028 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.449203 15028 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.451017 15024 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.451797 15024 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.454160 15024 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:06.289665 14974 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.878s	user 0.007s	sys 0.000s
I20260709 13:16:05.849262 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:06.456233 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 922) took 1210 ms. Trace:
I20260709 13:16:06.456703 11366 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:06.442139 11441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.271s	user 0.010s	sys 0.036s
I20260709 13:16:06.460215 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 788) took 1190 ms. Trace:
I20260709 13:16:06.478652 12068 rpcz_store.cc:276] 0709 13:16:05.245770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.245872 (+   102us) service_pool.cc:224] Handling call
0709 13:16:06.456172 (+1210300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:06.461246 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 791) took 1025 ms. Trace:
W20260709 13:16:06.296909 14989 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.106s	user 0.006s	sys 0.003s
I20260709 13:16:06.454989 15029 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.460661 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2104) took 1229 ms. Trace:
I20260709 13:16:06.479352 11364 rpcz_store.cc:276] 0709 13:16:05.270038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.270614 (+   576us) service_pool.cc:224] Handling call
0709 13:16:05.273506 (+  2892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.439351 (+1165845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.460107 (+ 20756us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:06.292609 14952 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.480041 11365 rpcz_store.cc:276] 0709 13:16:05.412511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.413384 (+   873us) service_pool.cc:224] Handling call
0709 13:16:06.437372 (+1023988us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:05.924249 14846 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:06.291317 14358 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.512458 14776 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:05.477399 14995 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.491863 11519 rpcz_store.cc:276] 0709 13:16:05.210170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.214366 (+  4196us) service_pool.cc:224] Handling call
0709 13:16:06.439248 (+1224882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:06.293241 14894 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.408562 14314 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:05.516690 14848 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:05.476652 14997 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:05.796329 14863 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:06.319669 15025 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:05.512256 14926 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:05.779927 14924 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.501734 11366 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:06.507202 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:06.494110 14989 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:06.496825 14871 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.461982 14638 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.507928 15026 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.554427 12068 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:06.509279 15024 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.554610 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 790) took 1266 ms. Trace:
I20260709 13:16:06.539899 14848 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.504117 15037 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.565970 11366 rpcz_store.cc:276] 0709 13:16:05.279983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.280221 (+   238us) service_pool.cc:224] Handling call
0709 13:16:06.546019 (+1265798us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:06.570055 15026 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.503748 14952 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.500875 15029 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.506954 14358 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.575309 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:06.574754 12068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 930) took 1300 ms (client timeout 1635 ms). Trace:
I20260709 13:16:06.548496 14813 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.587626 14813 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.588486 14905 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.588981 15043 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.564998 14851 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:06.590310 14851 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.590539 14997 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:06.509100 14776 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.592356 14851 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.592995 14776 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.595083 15029 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.595676 15029 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:06.596155 15029 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.598153 15029 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.581200 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.599702 11363 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:06.562496 15030 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.545198 15020 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.589530 15043 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.600742 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:06.619313 11363 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:06.549101 14314 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.583230 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.623935 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:06.601614 11409 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 1.567s (SENT) [suppressed 42 similar messages]
I20260709 13:16:06.623262 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.561832 14849 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:06.606048 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.561968 14924 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.628787 15033 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:06.629594 15033 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:06.630479 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 1.77229e+06us
I20260709 13:16:06.609254 14247 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:06.563041 14871 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.611734 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 792) took 1132 ms. Trace:
I20260709 13:16:06.612128 15034 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.612993 14827 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:06.613830 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.614518 14979 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:06.615245 14855 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:06.545770 14863 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.565498 14951 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.642179 14827 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:06.649145 15008 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.648960 14951 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.616433 15018 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.574494 15024 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.649411 14827 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.650610 14827 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.650985 14951 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.579001 14989 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.634387 14247 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:06.651641 14989 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.652586 14247 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.653601 14247 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.653717 14989 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.488286 14974 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.534037 14995 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.656767 15036 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:16:06.640007 15028 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.658712 14995 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.586839 11365 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:06.651412 15024 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.660091 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:06.661187 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.662293 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.664670 14863 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.623347 15030 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.667114 15030 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.625360 11368 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:06.562260 15038 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.536096 14894 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.578452 15031 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.658622 15028 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.628026 12033 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:06.598976 15032 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.506074 14846 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.628887 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.633769 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:06.685813 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 57843.4us
I20260709 13:16:06.611174 14899 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:06.639883 11367 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:06.635308 11369 rpcz_store.cc:276] 0709 13:16:05.479899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.480156 (+   257us) service_pool.cc:224] Handling call
0709 13:16:05.480533 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:05.852195 (+371662us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:06.343734 (+491539us) log.cc:844] Serialized 14 byte log entry
0709 13:16:06.509849 (+166115us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.510363 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:16:06.510499 (+   136us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:06.510533 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.510607 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.611478 (+100871us) raft_consensus.cc:1685] finished
0709 13:16:06.611500 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.611666 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":454178,"prepare.queue_time_us":594639,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":474989,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":9048,"thread_start_us":491263,"threads_started":3,"wal-append.queue_time_us":127216}
I20260709 13:16:06.631573 14924 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.550025 14926 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.646041 11364 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:06.631656 15033 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.693949 15028 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.617476 14952 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.584749 15037 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.549239 15025 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:06.667943 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 224989us
W20260709 13:16:06.587308 12068 rpcz_store.cc:269] 0709 13:16:05.274464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.491305 (+1216841us) service_pool.cc:224] Handling call
0709 13:16:06.574654 (+ 83349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:06.708581 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:06.663271 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.578055 14848 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.574110 14633 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.642441 14979 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:06.686092 11361 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:06.687659 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.644613 14855 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:06.688331 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.687294 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:06.692023 14994 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:06.693483 14871 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.659271 14995 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.654232 14989 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:06.634932 14849 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.632266 14314 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.703105 11440 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:06.707216 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.735391 11519 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:06.710044 11365 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:06.736881 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.695664 14899 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.740722 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:06.741426 11440 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.743780 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:06.713578 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 111994us
I20260709 13:16:06.625898 15020 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.694648 15030 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:06.751163 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:06.708282 14352 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.714740 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:06.714622 11368 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:06.753028 11436 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.741588 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:06.753541 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.720134 14358 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.716711 11516 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:06.720717 15044 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:06.699173 15024 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.762257 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.732218 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:06.763388 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.724960 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:06.763857 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.734002 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:06.764746 11439 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:06.706733 15048 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.754179 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 928) took 1480 ms. Trace:
I20260709 13:16:06.765925 12068 rpcz_store.cc:276] 0709 13:16:05.273960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.714864 (+1440904us) service_pool.cc:224] Handling call
0709 13:16:06.754114 (+ 39250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:06.756527 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 87460.7us
I20260709 13:16:06.758282 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.771857 11335 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:06.759876 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.760499 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:06.733004 14952 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:06.759876 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:06.698918 15043 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.773365 11437 raft_consensus.cc:2410] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:06.647490 15034 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.772797 11365 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:06.763022 11442 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:06.687860 15031 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:06.774515 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 49627.7us
I20260709 13:16:06.785259 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:06.774678 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:06.782083 15048 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.782804 11335 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:06.788367 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.791697 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:06.724536 14633 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.751452 14995 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.783695 14863 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:06.828634 14358 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.845939 14863 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.772631 15044 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.846160 14358 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.847828 15048 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.848155 15044 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.774115 14952 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.848829 14952 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:06.688812 15038 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.835479 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.851351 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 794) took 1025 ms. Trace:
I20260709 13:16:06.851990 11519 raft_consensus.cc:2410] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:06.852877 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:06.853410 11519 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.802366 11440 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:06.851763 11362 rpcz_store.cc:276] 0709 13:16:05.825929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.826515 (+   586us) service_pool.cc:224] Handling call
0709 13:16:05.826884 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.438048 (+611164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.438077 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.483303 (+ 45226us) log.cc:844] Serialized 26 byte log entry
0709 13:16:06.571379 (+ 88076us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:06.571440 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.571511 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.851124 (+279613us) raft_consensus.cc:1685] finished
0709 13:16:06.851144 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.851294 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121484,"raft.queue_time_us":610804,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6470,"thread_start_us":37324,"threads_started":1,"wal-append.queue_time_us":230735}
I20260709 13:16:06.858450 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:06.859215 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 67599.5us
W20260709 13:16:06.861800 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 70750.3us
I20260709 13:16:06.700294 15032 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.867748 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:06.872099 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:06.750265 14989 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.868693 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.874069 11362 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:06.875241 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.848492 14314 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.877254 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:06.791568 15043 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:06.879209 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.882356 11365 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:06.880539 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.881199 11369 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:06.884810 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:06.888211 11436 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:06.830237 15040 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:06.793357 14972 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:06.884016 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 148138us
I20260709 13:16:06.813779 14848 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.834877 14974 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.837527 15028 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.842162 15020 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.861557 14863 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.845129 14926 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.882157 15043 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.830426 15050 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:06.774087 15024 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.793678 15047 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:06.895223 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.869138 15048 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:06.915769 15048 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.916203 15048 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.916455 15049 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.912317 15043 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.867389 15036 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.844205 15026 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.897437 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.893004 14827 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:06.870795 14816 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:06.872615 14358 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.900983 11519 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
W20260709 13:16:06.901824 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 139728us
I20260709 13:16:06.942647 14846 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.901067 14972 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:06.788518 15030 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:16:06.903913 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:06.947005 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:06.947494 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:06.903501 12063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 914) took 2478 ms (client timeout 1888 ms). Trace:
I20260709 13:16:06.902923 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:06.829666 14849 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.874606 15008 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:16:06.843531 14855 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:06.904640 15055 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:16:06.915154 14994 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:06.913767 11365 raft_consensus.cc:2410] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:06.955765 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.957139 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.957304 15024 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.958210 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:06.927975 15036 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.959257 15032 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.960292 15036 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.898979 15040 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:06.961177 15040 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:06.917021 15049 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.939262 14827 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:06.941706 14899 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:06.789165 14979 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:06.917044 14633 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:06.948094 11442 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
W20260709 13:16:06.971285 15059 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:16:06.949221 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51015.7us
I20260709 13:16:06.952437 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:06.972370 11436 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 3.
I20260709 13:16:06.955999 14605 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:06.973174 14605 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:06.954703 14855 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.939821 11362 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:06.940418 15031 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:06.976293 14979 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:06.955624 14972 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:06.963181 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:06.962054 12069 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:16:06.979370 12069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 929) took 1705 ms (client timeout 2239 ms). Trace:
I20260709 13:16:06.965983 14827 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:06.947917 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:06.924518 14989 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:06.948127 12063 rpcz_store.cc:269] 0709 13:16:04.421797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.422676 (+   879us) service_pool.cc:224] Handling call
0709 13:16:06.900349 (+2477673us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54007783d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:06.900478 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126579072}
I20260709 13:16:06.931705 15047 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:06.979470 11365 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:06.965384 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:06.980007 12069 rpcz_store.cc:269] 0709 13:16:05.274193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.903698 (+1629505us) service_pool.cc:224] Handling call
0709 13:16:06.979311 (+ 75613us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:06.981839 11437 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:06.973383 14947 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:06.944000 15050 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:06.986518 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:06.989123 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:06.991104 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:06.991734 14358 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:06.970206 15053 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.997704 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.001647 12063 raft_consensus.cc:2410] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:06.998266 12069 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:06.981356 14605 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:07.002566 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:07.005371 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.005731 15031 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:07.003794 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 931) took 1609 ms. Trace:
I20260709 13:16:07.006031 12069 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:07.006259 12063 rpcz_store.cc:276] 0709 13:16:05.394262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.984099 (+1589837us) service_pool.cc:224] Handling call
0709 13:16:07.003736 (+ 19637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.009370 15031 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:07.009575 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.010666 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:07.005450 12034 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:07.015993 14247 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:07.016474 14247 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:07.017622 12034 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:07.010298 15031 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.006865 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:06.986881 15047 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.003818 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:07.028344 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:07.054328 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:07.055449 12069 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:06.999989 14358 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:07.021716 12063 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:07.007709 14947 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:07.060598 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:07.061798 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:07.067028 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:07.017755 15038 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.079106 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.054998 14994 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:07.078637 11440 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:07.080873 12063 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:07.055040 15053 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.062829 14951 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:07.062654 14358 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:07.189790 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.259070 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:07.340574 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 82373.6us
I20260709 13:16:07.191211 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:07.364914 11519 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:07.083986 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.346328 11368 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:07.065070 14947 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.083348 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:07.078064 14988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.307s	user 0.018s	sys 0.040s
I20260709 13:16:07.080511 14635 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:16:07.124234 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.367539 11443 raft_consensus.cc:2410] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:16:07.367699 14947 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:07.382274 12069 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:07.330432 14827 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:07.246871 15049 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.312810 15034 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.363927 14951 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:07.370517 15040 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.189363 14995 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.370270 15030 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.370275 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:07.372016 15022 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.052s	user 0.000s	sys 0.003s
I20260709 13:16:07.374048 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:07.192317 15057 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:16:07.247752 15050 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:07.376173 14988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.607s	user 0.037s	sys 0.055s
I20260709 13:16:07.382674 14635 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:07.355731 14994 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.311650 15026 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.083649 14247 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:07.524025 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 141496us
I20260709 13:16:07.682974 15030 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.688944 15030 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.333293 14605 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:07.690395 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 660) took 2521 ms. Trace:
I20260709 13:16:07.668171 15034 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:07.691195 14605 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.691612 15034 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:07.693775 15034 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.694274 15034 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:07.342082 14863 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.416545 15048 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:07.695465 12061 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:07.369701 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:07.203681 14952 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.574234 14951 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:07.582988 15064 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.690768 11441 rpcz_store.cc:276] 0709 13:16:05.169014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.169159 (+   145us) service_pool.cc:224] Handling call
0709 13:16:05.169546 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:05.169961 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.169992 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:05.170509 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:16:06.479031 (+1308522us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:06.479076 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.479154 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.371631 (+892477us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:07.690141 (+318510us) raft_consensus.cc:1685] finished
0709 13:16:07.690162 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.690336 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1048,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":17226,"spinlock_wait_cycles":378846976,"thread_start_us":1308193,"threads_started":1,"wal-append.queue_time_us":1149330}
I20260709 13:16:07.700237 14863 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.703655 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.700755 14994 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:07.679986 14833 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.708884 14635 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.712754 15048 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.702822 15026 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.716367 14827 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.715999 14994 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.718765 14994 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.718837 15053 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.707858 15064 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:07.721246 15043 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.804s	user 0.008s	sys 0.000s
I20260709 13:16:07.721575 15064 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:07.723775 14972 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.737s	user 0.006s	sys 0.003s
I20260709 13:16:07.724182 14952 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.724908 12068 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:07.694685 15034 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.734477 11519 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:07.736500 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.737610 11367 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:07.738587 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.739502 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.740693 14899 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.741575 15034 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.741766 14813 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:07.742935 15047 consensus_meta.cc:287] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.677s	user 0.005s	sys 0.004s
I20260709 13:16:07.750746 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.751689 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.753400 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.755463 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.761206 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.743379 15047 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.724222 14972 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.770934 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.721760 15043 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.753953 15026 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.771865 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.774783 11367 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:07.775686 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.766739 14799 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.785737 11362 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:07.793449 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.793823 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:07.794400 12063 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:16:07.794785 15026 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:07.795552 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.782732 15037 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.798992 15040 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.806166 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:07.845216 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.848268 15047 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.849726 14972 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.853811 14951 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.856489 14358 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.858242 14247 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.858788 14247 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:07.859349 15063 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:16:07.864308 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 809) took 1138 ms. Trace:
I20260709 13:16:07.864692 11365 rpcz_store.cc:276] 0709 13:16:06.725479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.995936 (+270457us) service_pool.cc:224] Handling call
0709 13:16:06.996289 (+   353us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:07.005026 (+  8737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.005056 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:07.005637 (+   581us) log.cc:844] Serialized 52 byte log entry
0709 13:16:07.079738 (+ 74101us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:07.250254 (+170516us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.256070 (+  5816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.256147 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.864051 (+607904us) raft_consensus.cc:1685] finished
0709 13:16:07.864074 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.864230 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8363,"prepare.queue_time_us":111680,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":15360,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":171,"thread_start_us":136535,"threads_started":2,"wal-append.queue_time_us":745388}
I20260709 13:16:07.893603 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.894719 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.896176 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.899201 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.900247 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1729) took 1154 ms. Trace:
I20260709 13:16:07.900683 11368 rpcz_store.cc:276] 0709 13:16:06.746102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.758062 (+ 11960us) service_pool.cc:224] Handling call
0709 13:16:07.900104 (+1142042us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54002db754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:07.900195 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320791424}
I20260709 13:16:07.901863 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:07.902395 11368 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:16:07.918344 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.920347 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 934) took 1220 ms. Trace:
I20260709 13:16:07.920722 12063 rpcz_store.cc:276] 0709 13:16:06.699811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.828557 (+1128746us) service_pool.cc:224] Handling call
0709 13:16:07.920289 (+ 91732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.858659 14358 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.932825 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1128) took 1069 ms. Trace:
I20260709 13:16:07.933215 12061 rpcz_store.cc:276] 0709 13:16:06.863276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.373815 (+510539us) service_pool.cc:224] Handling call
0709 13:16:07.932663 (+558848us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b540070f3d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:07.932762 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33762560}
I20260709 13:16:07.851320 15043 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.933962 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.934070 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:07.935424 12061 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:07.937724 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:07.937942 11514 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2087) took 3032 ms (client timeout 2073 ms). Trace:
I20260709 13:16:07.939111 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:07.938428 11514 rpcz_store.cc:269] 0709 13:16:04.904138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.908945 (+  4807us) service_pool.cc:224] Handling call
0709 13:16:07.936332 (+3027387us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540031ea54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:07.936456 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299608320}
I20260709 13:16:07.940151 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:07.941442 11365 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:07.941982 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 816) took 1048 ms. Trace:
I20260709 13:16:07.942384 11365 rpcz_store.cc:276] 0709 13:16:06.893975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.941012 (+1047037us) service_pool.cc:224] Handling call
0709 13:16:07.941283 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.941793 (+   510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.941935 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.943562 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.944125 14947 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.945271 11365 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:07.946148 14947 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.946592 11365 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:07.944131 12066 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:07.947571 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.947820 11365 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:07.948072 12066 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:07.948077 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2109) took 1375 ms. Trace:
I20260709 13:16:07.950385 11512 rpcz_store.cc:276] 0709 13:16:06.572515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.572667 (+   152us) service_pool.cc:224] Handling call
0709 13:16:06.572948 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.594044 (+ 21096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.594073 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.594114 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:06.594148 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.947887 (+1353739us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5400150ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:07.947927 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.948027 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48049920}
I20260709 13:16:07.946499 12063 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:07.952032 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 936) took 1241 ms. Trace:
I20260709 13:16:07.952255 12063 rpcz_store.cc:276] 0709 13:16:06.710600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.935106 (+1224506us) service_pool.cc:224] Handling call
0709 13:16:07.951982 (+ 16876us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.952966 14247 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.952771 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:07.954993 15070 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.955353 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 923) took 2709 ms. Trace:
I20260709 13:16:07.956642 12066 rpcz_store.cc:276] 0709 13:16:05.246047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.954251 (+2708204us) service_pool.cc:224] Handling call
0709 13:16:07.954517 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.954799 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.954814 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.954840 (+    26us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:07.954857 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.954889 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.954957 (+    68us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.953867 14951 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:07.972836 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 925) took 2708 ms. Trace:
I20260709 13:16:07.975569 12066 rpcz_store.cc:276] 0709 13:16:05.264765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.971912 (+2707147us) service_pool.cc:224] Handling call
0709 13:16:07.972451 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.972578 (+   127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.972593 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.972642 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:07.972675 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.972713 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.972798 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.976837 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 926) took 2703 ms. Trace:
I20260709 13:16:07.977113 12066 rpcz_store.cc:276] 0709 13:16:05.273402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.976224 (+2702822us) service_pool.cc:224] Handling call
0709 13:16:07.976489 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.976614 (+   125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.976628 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.976652 (+    24us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:07.976670 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.976700 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.976780 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.978252 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 686) took 1403 ms. Trace:
I20260709 13:16:07.978469 12066 rpcz_store.cc:276] 0709 13:16:06.575338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.977724 (+1402386us) service_pool.cc:224] Handling call
0709 13:16:07.977967 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.978077 (+   110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.978090 (+    13us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.978113 (+    23us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:07.978130 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.978159 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.978218 (+    59us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.979243 12066 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:07.979635 11368 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:07.979748 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 933) took 1284 ms. Trace:
I20260709 13:16:07.983577 12066 rpcz_store.cc:276] 0709 13:16:06.695718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.978976 (+1283258us) service_pool.cc:224] Handling call
0709 13:16:07.979141 (+   165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.979538 (+   397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.979701 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.984902 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 935) took 1278 ms. Trace:
I20260709 13:16:07.985213 12066 rpcz_store.cc:276] 0709 13:16:06.706338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.984265 (+1277927us) service_pool.cc:224] Handling call
0709 13:16:07.984495 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.984643 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.984665 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.984700 (+    35us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:07.984727 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.984782 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.984862 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.986426 12066 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:07.987142 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 937) took 1271 ms. Trace:
I20260709 13:16:07.987527 12066 rpcz_store.cc:276] 0709 13:16:06.715542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.986032 (+1270490us) service_pool.cc:224] Handling call
0709 13:16:07.986275 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.986957 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.987102 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.988340 14998 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:07.988588 12066 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:07.988833 15026 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:07.989177 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 938) took 1267 ms. Trace:
I20260709 13:16:07.989368 15034 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:07.989527 12066 rpcz_store.cc:276] 0709 13:16:06.721307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.988225 (+1266918us) service_pool.cc:224] Handling call
0709 13:16:07.988443 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:07.988979 (+   536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.989136 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:07.990294 15034 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:07.992326 15034 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.993276 14998 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:07.970844 15050 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:07.982788 14923 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:07.995553 15050 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:07.995573 15026 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:07.996433 11516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1637) took 1308 ms (client timeout 1717 ms). Trace:
W20260709 13:16:07.996803 11516 rpcz_store.cc:269] 0709 13:16:06.687674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.763168 (+ 75494us) service_pool.cc:224] Handling call
0709 13:16:07.996260 (+1233092us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000e9f54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:07.996381 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391614208}
I20260709 13:16:07.997630 15050 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.998076 15026 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.999732 14804 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:07.995131 14998 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.000882 12061 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:07.989614 14811 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.004623 14811 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.006709 14811 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:07.969605 15070 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.009845 15070 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:08.017323 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:08.018143 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.021552 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.022454 11365 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:16:08.021450 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.026386 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.022339 11334 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.028517 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.030854 12033 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:08.032721 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.033921 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.038503 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.039088 11439 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:08.039433 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.040102 11444 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:08.039968 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.040521 11439 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:08.041158 11440 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:16:08.043509 11437 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.051079 14947 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:08.052157 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.052803 11443 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:08.054005 15041 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:08.054437 15041 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:08.054762 15041 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:08.059271 11445 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:08.061709 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.062337 11441 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:08.037915 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.064713 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.065943 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.071076 11368 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:08.060364 14770 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:08.074714 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.063827 14635 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:16:08.074795 14770 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:08.074645 14899 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:08.075907 14899 consensus_queue.cc:1243] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:16:08.074807 14776 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.076267 14811 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:08.076783 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.077683 11365 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:08.077127 11442 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.079356 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.029562 15070 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:08.085237 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.086009 11441 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:08.083949 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.076046 14635 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:08.089385 11365 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:08.092446 14247 consensus_queue.cc:1059] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:08.076119 14770 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.086979 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.096740 11445 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:08.080987 15061 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:08.101027 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.102239 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.102837 11441 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 4: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:08.103891 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.104460 11443 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:08.105597 14894 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:08.108522 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:08.109148 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.110968 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:08.111191 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.074831 14983 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:16:08.076630 14776 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.112062 11439 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:16:08.112340 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:08.112900 11510 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.113149 11439 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:08.114708 14776 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.116523 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:08.117076 11514 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.117347 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.117898 11445 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:08.118002 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.118602 11437 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.120224 11442 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:08.111703 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.123574 11515 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:08.124751 11512 raft_consensus.cc:2376] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10. Re-sending same reply.
I20260709 13:16:08.118325 11438 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:08.128422 11510 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:08.080351 14811 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:08.085966 14947 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:16:08.090766 14635 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:08.139480 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.133375 14811 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:08.142122 15075 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:16:08.145383 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.149901 11437 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:08.163767 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.164491 11437 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:16:08.101727 15061 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:08.155566 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.155975 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.168516 11440 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:08.158600 11514 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:08.155589 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.162156 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.162575 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.099620 15070 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:08.338380 12068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 927) took 3064 ms (client timeout 2209 ms). Trace:
W20260709 13:16:08.340886 12068 rpcz_store.cc:269] 0709 13:16:05.273712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.859727 (+1586015us) service_pool.cc:224] Handling call
0709 13:16:08.338175 (+1478448us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54006d56d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:08.338299 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930576640}
I20260709 13:16:08.337630 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1719) took 1761 ms. Trace:
W20260709 13:16:08.342595 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.343071 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:08.344007 11511 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:08.344657 11512 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.342355 11364 rpcz_store.cc:276] 0709 13:16:06.576575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.576728 (+   153us) service_pool.cc:224] Handling call
0709 13:16:06.591556 (+ 14828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.700056 (+108500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.337299 (+1637243us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b5400423954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:08.337552 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124588800}
I20260709 13:16:08.345891 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.271126 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1133) took 1315 ms. Trace:
I20260709 13:16:08.346714 12061 rpcz_store.cc:276] 0709 13:16:06.955621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.001497 (+1045876us) service_pool.cc:224] Handling call
0709 13:16:08.001770 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.002105 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:08.002130 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:08.002547 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.159559 (+157012us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:08.159605 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.159703 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.270877 (+111174us) raft_consensus.cc:1685] finished
0709 13:16:08.270917 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.271060 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":4540,"thread_start_us":156688,"threads_started":1,"wal-append.queue_time_us":252199}
W20260709 13:16:08.266423 12069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 688) took 1519 ms (client timeout 1954 ms). Trace:
W20260709 13:16:08.368019 12069 rpcz_store.cc:269] 0709 13:16:06.746848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.059632 (+312784us) service_pool.cc:224] Handling call
0709 13:16:08.266099 (+1206467us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b54005127d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:08.266343 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36841728}
W20260709 13:16:08.213460 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.174723 11439 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:16:08.266973 12063 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:08.354200 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.224445 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.252058 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:08.340988 11437 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:08.380125 11436 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:08.194303 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.382758 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:08.419715 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50048.8us
I20260709 13:16:08.252974 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.232131 11266 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 3, term changed from 3 to 5, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to <none>, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65), VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66), VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67), VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:08.566666 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1718) took 1993 ms. Trace:
I20260709 13:16:08.480522 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:08.264900 15021 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.943s	user 0.006s	sys 0.011s
I20260709 13:16:08.161240 15036 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:08.573572 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.421622 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.460565 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.568617 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.549026 11436 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:08.598750 12061 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:08.597607 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:08.579909 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.600893 11518 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:08.601078 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.569895 11515 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:08.577054 11266 catalog_manager.cc:5697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.596954 14783 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.599154 11364 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.578323 12068 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
W20260709 13:16:08.598480 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.601773 12061 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:08.601850 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.612139 11518 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:08.609930 11266 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
W20260709 13:16:08.577154 15041 consensus_meta.cc:287] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 13:16:08.569620 11366 rpcz_store.cc:276] 0709 13:16:06.573274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.574191 (+   917us) service_pool.cc:224] Handling call
0709 13:16:06.574466 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.591310 (+ 16844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.591339 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.591377 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:06.591426 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.566390 (+1974964us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5400166a54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:08.566437 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.566601 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38292224}
I20260709 13:16:08.169464 14811 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.558900 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.557494 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.572402 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:08.493937 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.605010 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:08.486719 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 368907us
I20260709 13:16:08.592762 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:08.605732 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.606612 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.215734 14864 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:08.607577 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:08.550554 15023 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 2.230s	user 0.011s	sys 0.012s
W20260709 13:16:08.608474 11362 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2259) took 1619 ms (client timeout 1524 ms). Trace:
I20260709 13:16:08.570583 14983 consensus_queue.cc:1243] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:08.609325 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1720) took 2022 ms. Trace:
I20260709 13:16:08.652691 11369 rpcz_store.cc:276] 0709 13:16:06.586992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.715016 (+128024us) service_pool.cc:224] Handling call
0709 13:16:06.742749 (+ 27733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.884438 (+141689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.609108 (+1724670us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54000a73d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:08.609267 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312477568}
I20260709 13:16:08.612954 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.653461 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.654013 11518 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:08.607156 14783 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.650123 14811 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.652648 14835 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:08.657759 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.658330 11518 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:08.659117 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.660109 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:08.661044 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.661969 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.662909 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.614624 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:08.663774 12068 raft_consensus.cc:2376] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13. Re-sending same reply.
I20260709 13:16:08.617055 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:08.664772 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 47788.2us
I20260709 13:16:08.617683 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 903) took 5076 ms. Trace:
I20260709 13:16:08.618312 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.666501 11436 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:08.666276 12067 rpcz_store.cc:276] 0709 13:16:03.541491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.542404 (+   913us) service_pool.cc:224] Handling call
0709 13:16:03.542743 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.543073 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.543098 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:05.146712 (+1603614us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b2c000a6788. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:05.150387 (+  3675us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.235557 (+ 85170us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:05.235593 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.235662 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.264220 (+3028558us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:08.617469 (+353249us) raft_consensus.cc:1685] finished
0709 13:16:08.617489 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.617611 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3417,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5216,"spinlock_wait_cycles":387010688,"thread_start_us":70248,"threads_started":1,"wal-append.queue_time_us":1156676}
I20260709 13:16:08.656956 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.668823 11369 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.621182 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:08.621896 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.624537 15070 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.671543 12061 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:08.665885 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:08.625591 11364 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:08.256547 14635 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:08.651350 11362 rpcz_store.cc:269] 0709 13:16:06.988581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.111256 (+122675us) service_pool.cc:224] Handling call
0709 13:16:08.608293 (+1497037us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54000a78d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:08.608410 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966023424}
I20260709 13:16:08.673847 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:08.652192 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 59532.4us
I20260709 13:16:08.653198 12065 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 4: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:08.630003 11266 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 7 to 8, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 8 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.675268 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.676612 11266 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 6, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:08.655575 11515 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:08.558619 14775 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.651662 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.674829 12068 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.680313 11366 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.651687 11512 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:08.676244 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 679) took 3662 ms. Trace:
I20260709 13:16:08.655300 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.683360 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.685380 11515 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:08.676466 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.675298 12060 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:08.686269 11363 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:08.681901 11266 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:08.672930 11518 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:16:08.676376 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.674635 14825 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.689842 11266 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 7, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 7 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.256191 15061 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.687234 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.695840 11436 raft_consensus.cc:2368] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:08.683059 12062 rpcz_store.cc:276] 0709 13:16:05.014093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.015565 (+  1472us) service_pool.cc:224] Handling call
0709 13:16:05.015887 (+   322us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:05.016252 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.016277 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:05.031210 (+ 14933us) log.cc:844] Serialized 52 byte log entry
0709 13:16:05.226777 (+195567us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:05.226819 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.226884 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.485822 (+3258938us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:08.664475 (+178653us) raft_consensus.cc:1685] finished
0709 13:16:08.664496 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.676176 (+ 11680us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4999,"raft.queue_time_us":4401,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":310823808,"thread_start_us":147748,"threads_started":1,"wal-append.queue_time_us":1259478}
I20260709 13:16:08.692739 14825 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:08.704641 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:08.705861 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 49330.7us
I20260709 13:16:08.694504 11266 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 1 to 5, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 5 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.708102 14851 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:08.708513 14851 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:08.709205 14851 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.708899 11266 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 2 to 4, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65), VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66), VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67), VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:08.710388 14851 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.711210 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.688735 11364 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:08.711814 11517 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:08.690603 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.712447 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.712793 14825 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:08.702418 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.710628 11266 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 7, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:08.713496 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.714251 11362 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:16:08.714392 11266 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 6, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 6 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.702970 14811 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.706594 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:08.715103 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.698028 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.720773 12068 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:08.720731 11510 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:08.721258 11366 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:08.722200 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.713471 11436 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:08.716274 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:08.723126 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.675881 14835 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:08.702752 14783 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.717523 11512 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:08.717151 11266 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.715798 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:08.718405 11369 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:08.713511 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.727514 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:08.728379 14835 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.728860 14835 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:08.730901 11510 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:08.650434 15041 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:08.731755 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.731972 11369 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.732688 14783 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:08.680953 14864 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.725092 11366 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:08.717537 11440 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:16:08.734356 11440 pending_rounds.cc:85] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 7
I20260709 13:16:08.734597 11440 pending_rounds.cc:107] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:16:08.734886 11266 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 6, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 6 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.736778 11266 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 7, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 7 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.734827 11440 raft_consensus.cc:2889] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:08.738440 11266 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.733245 14783 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.739010 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.739670 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:08.739936 11440 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:08.690433 14998 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:08.742065 11515 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:08.747895 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.748824 11440 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:08.748190 11266 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.746730 11361 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:08.749935 11266 catalog_manager.cc:5697] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:08.749596 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.751107 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.752662 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.690680 15070 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:16:08.734772 14864 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.751579 11266 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 7, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 7 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.753254 11365 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:08.747392 14811 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:16:08.747603 14983 consensus_queue.cc:1059] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:16:08.749115 15081 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:08.767616 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 641158us
I20260709 13:16:08.759649 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:08.765443 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 665515us
W20260709 13:16:08.770915 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.775681 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 674) took 2025 ms. Trace:
I20260709 13:16:08.776054 11438 rpcz_store.cc:276] 0709 13:16:06.750783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.750901 (+   118us) service_pool.cc:224] Handling call
0709 13:16:08.775492 (+2024591us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400438ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:08.775626 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3142272768}
I20260709 13:16:08.777168 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.777349 11266 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 3 to 6, term changed from 5 to 6, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to <none>, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 6 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:08.778946 11266 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 2 to 3, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:08.754959 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:08.764225 14247 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:08.748044 14998 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:08.756582 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.790531 11440 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:08.732975 15041 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.768761 11335 leader_election.cc:400] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:08.792923 11335 leader_election.cc:403] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:08.759235 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:08.795531 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.758397 14901 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:08.765141 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.769721 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 924) took 3523 ms. Trace:
I20260709 13:16:08.763726 15070 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:08.771420 14783 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:08.755862 11443 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.045s	sys 0.058s
I20260709 13:16:08.774230 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.785215 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1716) took 4597 ms. Trace:
I20260709 13:16:08.800288 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.761444 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1129) took 1897 ms. Trace:
W20260709 13:16:08.811474 11443 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.045s	sys 0.059s
I20260709 13:16:08.801656 12063 rpcz_store.cc:276] 0709 13:16:05.246292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.958423 (+2712131us) service_pool.cc:224] Handling call
0709 13:16:08.266684 (+308261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.378972 (+112288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.769550 (+390578us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54005127d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:08.769669 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682435840}
I20260709 13:16:08.801586 11442 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:16:08.770539 14864 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:08.796782 14848 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:08.810987 12066 rpcz_store.cc:276] 0709 13:16:06.863591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.990391 (+1126800us) service_pool.cc:224] Handling call
0709 13:16:07.990647 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.002729 (+ 12082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.002756 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:08.046514 (+ 43758us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.084074 (+ 37560us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:08.096954 (+ 12880us) log.cc:844] Serialized 14 byte log entry
0709 13:16:08.760862 (+663908us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b18003471a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:16:08.761047 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.761119 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.761159 (+    40us) raft_consensus.cc:1685] finished
0709 13:16:08.761256 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.761388 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2949,"prepare.queue_time_us":15473,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":666898,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":177,"spinlock_wait_cycles":48108928,"thread_start_us":37189,"threads_started":1,"wal-append.queue_time_us":192862}
I20260709 13:16:08.784447 11266 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 6, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 6 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.820691 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.809356 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.823599 14638 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:08.791365 14247 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:08.794042 14998 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.829059 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.840329 11436 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:08.809783 11370 rpcz_store.cc:276] 0709 13:16:04.166452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.166912 (+   460us) service_pool.cc:224] Handling call
0709 13:16:04.167225 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:04.234813 (+ 67588us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:04.459126 (+224313us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.797908 (+338782us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b18004a5d20. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:05.267973 (+470065us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b18004a5d20. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf 00007f35bca18038
0709 13:16:05.271322 (+  3349us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.292068 (+ 20746us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.292135 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:05.292184 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.252867 (+2960683us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b18004a5d20. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419
0709 13:16:08.763172 (+510305us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b54000f91d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:08.763216 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.763342 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":823611,"prepare.queue_time_us":1043748,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":10577,"raft.queue_time_us":122234,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":111234,"spinlock_wait_cycles":446646016,"thread_start_us":283790,"threads_started":2,"wal-append.queue_time_us":226482,"wal-append.run_cpu_time_us":2368,"wal-append.run_wall_time_us":1584866}
W20260709 13:16:08.841383 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 164077us
I20260709 13:16:08.831349 11266 catalog_manager.cc:5697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 5, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 5 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:08.842288 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:08.846830 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.847415 11442 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.821547 14864 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:08.848527 14864 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:08.849489 15082 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:08.849699 14864 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.824162 14638 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:08.805719 15070 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.806838 15041 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:08.850992 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:08.844319 11266 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 4 to 5, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 5 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.914525 11266 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:08.861513 11442 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:08.916491 11442 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:08.821165 15080 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:08.882943 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:08.912431 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:08.839903 14783 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:08.919310 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:08.781934 15081 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.913966 15070 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:08.802455 14835 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.941285 14835 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:08.942595 14835 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:08.942927 15036 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:08.845430 14849 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:08.943595 15036 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:08.943779 14849 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:08.944378 14633 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:08.944747 14633 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:08.944705 14849 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:08.937741 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 86831.4us
I20260709 13:16:08.946951 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:08.947853 14813 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:08.948228 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:08.948900 14770 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:08.943009 14835 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:08.845053 15025 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:08.951723 14835 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.956912 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.957566 11437 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:16:08.958715 11437 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:08.966842 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 290530us
I20260709 13:16:08.967756 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:08.961964 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:08.882704 14848 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:08.969726 11436 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:08.970371 11442 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:08.970501 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:08.840224 14983 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:16:08.913239 15041 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.936672 14783 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:08.936652 14638 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:08.938046 14855 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:08.988283 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:08.988796 14848 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.000015 14848 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:09.000725 14638 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:08.941627 15019 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 13:16:09.005069 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 195760us
I20260709 13:16:09.010241 15061 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:09.011731 15086 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.012295 15086 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.012413 14848 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.014971 15086 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.015506 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.016130 11436 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:08.996203 15041 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.000006 14783 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.020789 14998 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.024471 15086 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:08.989144 14247 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.047778 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:09.034768 14783 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:09.051589 15087 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:09.052065 15087 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.052400 15087 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:09.055608 15088 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:09.056164 15088 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:09.057003 15088 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.070706 11436 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:09.063192 14835 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.065163 14848 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.079780 12064 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:09.087337 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.092562 14998 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:09.026731 15070 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.049077 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:09.015198 14633 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.101850 14804 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:09.066660 15086 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:09.091930 14635 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:09.085041 14852 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.086905 14855 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:09.083710 14775 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.087604 15081 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.065407 15036 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:09.115245 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 342350us
I20260709 13:16:09.156626 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:09.106467 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.107523 15086 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.153066 15070 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.116820 14852 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.160861 12034 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:09.162705 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 56320.2us
I20260709 13:16:09.109685 14855 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:09.179219 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:09.179643 12034 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.181078 14700 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:09.159991 15086 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:09.161338 14952 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:09.166404 15095 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:09.161764 14633 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:09.195518 14855 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.157387 14775 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.197793 14852 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.182822 14947 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:16:09.194657 14956 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:09.195088 14952 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:09.202255 15095 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.186430 14804 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:16:09.204629 14247 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:09.186949 15087 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.200388 15041 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.213991 14956 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.232460 14811 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:09.368220 14811 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:09.369719 14775 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.237648 14835 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:09.393468 14855 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.397662 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.301254 11368 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:09.398844 11365 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:09.304949 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:09.044879 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 256096us
W20260709 13:16:09.306454 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.405333 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.310521 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 688) took 1198 ms. Trace:
I20260709 13:16:09.407033 11443 rpcz_store.cc:276] 0709 13:16:08.109110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.126609 (+ 17499us) service_pool.cc:224] Handling call
0709 13:16:08.132762 (+  6153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.132982 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:08.812043 (+679061us) log.cc:844] Serialized 14 byte log entry
0709 13:16:09.306946 (+494903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:09.307089 (+   143us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:09.307222 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.307382 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.307773 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25173,"prepare.queue_time_us":82955,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":63765,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":10644,"thread_start_us":1137109,"threads_started":2,"wal-append.queue_time_us":420829}
I20260709 13:16:09.369897 14811 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.410323 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:09.411221 14811 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:09.310428 11445 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:09.413048 14775 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:09.413552 14775 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.414022 14775 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.414647 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2420) took 1275 ms. Trace:
W20260709 13:16:09.319216 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 330849us
I20260709 13:16:09.415134 11439 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:09.329953 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:09.332419 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:09.321455 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 316702us
I20260709 13:16:09.340152 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:09.345911 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:09.354185 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 752292us
I20260709 13:16:09.223912 14952 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:09.244721 14956 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:09.415082 11445 rpcz_store.cc:276] 0709 13:16:08.139091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.139210 (+   119us) service_pool.cc:224] Handling call
0709 13:16:09.414329 (+1275119us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54003151d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:09.414554 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2648749824}
W20260709 13:16:09.295445 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 495668us
I20260709 13:16:09.214361 15090 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:09.431393 15098 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.224155 15095 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:09.382930 15041 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.432083 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.434185 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:09.434281 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.387255 15036 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.435623 15088 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:09.236284 15087 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:09.430279 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 100807us
I20260709 13:16:09.436532 15088 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:09.437093 15088 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:09.438373 15088 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.439949 15097 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.440582 15097 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.442847 15097 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:09.443607 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.426174 11408 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.445343 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:09.446081 15090 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:09.444653 14998 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.447041 15093 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:09.447377 15090 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.449532 14813 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:09.450098 15093 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.450663 15093 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:09.447664 15087 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.450155 14813 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:09.445863 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.447052 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.444677 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 825) took 1420 ms. Trace:
I20260709 13:16:09.445961 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:09.452513 11368 rpcz_store.cc:276] 0709 13:16:08.023889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.095543 (+ 71654us) service_pool.cc:224] Handling call
0709 13:16:08.095826 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.398768 (+1302942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.415624 (+ 16856us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002db254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:09.444596 (+ 28972us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2769616640}
I20260709 13:16:09.457474 15097 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:09.457921 15097 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.458441 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:09.458284 15097 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.460320 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.461434 15070 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:09.462530 15107 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:09.471325 14845 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:09.471809 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 275997us
I20260709 13:16:09.473022 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.474120 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.471346 15087 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.478605 15087 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.470342 15025 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:09.480562 14813 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.468918 15107 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.483985 15107 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:09.488296 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 308619us
I20260709 13:16:09.488598 15041 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.488173 14845 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:09.490674 14845 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.460240 14956 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.493685 14894 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:09.494881 14956 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.460644 14952 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.497485 14952 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.512264 11365 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:09.516032 11368 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:16:09.517539 15025 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:09.518291 15025 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:09.518395 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.516054 15099 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:09.520536 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 46716.2us
I20260709 13:16:09.471563 14998 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.520258 15099 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.523223 14998 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.525074 15099 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:09.525454 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 52674.1us
I20260709 13:16:09.525934 14247 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:09.526350 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.533902 12034 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.532953 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.535425 14952 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:09.519491 15025 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.535775 15087 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:09.527379 14956 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:09.536334 15087 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.536860 15087 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.537127 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.532850 15113 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:09.541065 14247 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:09.537958 14956 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.541824 14956 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:09.541260 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 94390.2us
W20260709 13:16:09.543352 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 97514.9us
I20260709 13:16:09.545974 11333 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.548303 15113 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:09.560743 14635 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:09.562633 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:09.564630 14638 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:09.565299 14638 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:09.566574 14638 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.567699 15114 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:09.568204 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1139) took 1597 ms. Trace:
I20260709 13:16:09.568362 14998 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:09.570501 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.583266 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.556833 14894 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.585163 14635 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:09.581041 12067 rpcz_store.cc:276] 0709 13:16:07.970643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.676025 (+705382us) service_pool.cc:224] Handling call
0709 13:16:08.694576 (+ 18551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:08.778273 (+ 83697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.778314 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:08.955250 (+176936us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b2c000a6788. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:08.955864 (+   614us) log.cc:844] Serialized 52 byte log entry
0709 13:16:09.208644 (+252780us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:09.533129 (+324485us) log.cc:844] Serialized 14 byte log entry
0709 13:16:09.548639 (+ 15510us) log.cc:844] Serialized 14 byte log entry
0709 13:16:09.567531 (+ 18892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.567638 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:09.567769 (+   131us) raft_consensus.cc:1685] finished
0709 13:16:09.567781 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.567968 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100333,"prepare.queue_time_us":54669,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":109871,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":17792,"spinlock_wait_cycles":34846336,"thread_start_us":552008,"threads_started":2,"wal-append.queue_time_us":300223}
W20260709 13:16:09.597286 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 60345.1us
I20260709 13:16:09.599104 11513 raft_consensus.cc:2368] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10.
I20260709 13:16:09.588582 14952 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.599980 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:09.612193 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51799.5us
I20260709 13:16:09.616179 14635 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.620347 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.620586 15086 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:09.626536 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47133.6us
I20260709 13:16:09.635200 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.629001 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:09.636454 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.555394 15113 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.640549 15095 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.642228 14852 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:09.632717 15025 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.642915 14852 consensus_queue.cc:1243] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:09.635985 11514 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:09.629947 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.634944 14775 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.635097 11518 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:09.595829 15088 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:16:09.652652 15097 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.655586 15097 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.656332 11512 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:09.659049 14775 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.660543 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.661799 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.662716 14811 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.663456 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.663995 11511 raft_consensus.cc:2376] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10. Re-sending same reply.
I20260709 13:16:09.664947 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.665714 15107 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.640578 15087 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.642692 15095 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.636049 15092 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.673254 15107 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:09.675659 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45843.6us
W20260709 13:16:09.675554 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55348.4us
I20260709 13:16:09.671136 15114 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.643441 15025 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.687559 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.687723 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:09.688248 11517 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.688707 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.689301 11510 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:09.690677 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.683099 14811 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:09.691673 14811 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.692082 14811 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.692893 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:09.693660 11439 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:16:09.694644 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.695273 11445 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:09.695909 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:09.696108 14952 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.696741 15087 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:09.697347 15087 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:09.697628 15025 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.698310 14855 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:09.699950 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:09.701120 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.702831 11443 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:16:09.704025 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.704958 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.705334 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.705521 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.707998 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.708992 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:09.709542 11510 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:09.710212 11409 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.710656 11438 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:09.711148 14952 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.684607 15092 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.712401 11440 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:09.713909 15092 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.716223 15093 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.704043 15086 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.718532 15093 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.721887 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:09.724406 11409 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.724779 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.725749 15086 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.726320 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.726565 15092 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:09.727707 14775 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:09.728363 14775 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.729075 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.728890 14775 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:09.740873 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.742990 11445 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:09.751798 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:09.759799 11409 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.754719 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:09.761646 11439 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:09.742357 15093 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:09.762125 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.766929 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.768980 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.769162 14952 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:09.769197 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.771560 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.772159 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.769469 11438 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 13:16:09.768895 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.743630 15087 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:09.775767 11512 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:09.742347 15092 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.777034 11516 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:09.777362 14852 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:09.763521 15093 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.778337 15093 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.751617 14783 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:09.781329 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.784147 14855 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:09.784322 11333 leader_election.cc:400] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10.
I20260709 13:16:09.784605 14855 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.785056 14855 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.772755 14952 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.787694 14952 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:09.773612 15025 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:09.791559 15132 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:16:09.784685 11333 leader_election.cc:403] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:09.793011 14813 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:09.794025 15041 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:09.797484 15092 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.799574 15092 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.800172 14811 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.802464 14811 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:09.803104 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 76886.7us
W20260709 13:16:09.803563 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 43860us
I20260709 13:16:09.791870 15025 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:09.804944 15025 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.782691 11335 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.808468 11517 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:09.809603 11511 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:09.811765 14845 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:09.812493 14845 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:09.813059 14633 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:09.813459 14845 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.817051 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:09.817572 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88581.4us
I20260709 13:16:09.819384 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:09.819604 14633 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:09.819869 14894 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:09.819295 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.820318 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:09.820861 11437 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.823067 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:09.823786 11365 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:09.825136 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.826066 14952 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.826453 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.828030 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.828346 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:09.828979 12064 raft_consensus.cc:2376] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9. Re-sending same reply.
I20260709 13:16:09.829084 12066 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:09.829723 12067 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:09.830469 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:09.833197 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.833539 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.835135 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.838433 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.811762 14956 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.840653 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.841370 11512 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:09.819466 12032 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:09.842852 15114 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:09.843303 15114 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.843698 15114 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:09.842806 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.844543 11512 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:09.845810 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.846379 11516 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:09.848452 14775 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:09.849675 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.803898 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:09.835021 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.858506 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:09.859189 12066 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:09.820238 14633 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:09.864158 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.864717 11516 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.866954 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.870951 11437 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:09.856988 11441 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:09.865418 11519 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:09.806437 14813 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10.
I20260709 13:16:09.841866 14956 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.888116 11437 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:09.888952 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:09.889699 11443 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:09.892709 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 57755.7us
I20260709 13:16:09.891645 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:09.819604 15087 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:09.906306 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:09.910550 15092 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:09.915410 15093 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.917795 15093 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:09.915509 12066 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:09.888422 15086 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:09.928877 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 125045us
I20260709 13:16:09.930642 11368 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:09.909390 15087 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:09.908932 14633 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.940004 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.942623 11519 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.943084 12067 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:09.927728 14952 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.949091 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:09.950155 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 152933us
I20260709 13:16:09.960654 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:09.962169 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.935807 15107 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:09.953806 15114 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:09.970074 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:09.970216 15114 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:09.971050 14811 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:16:09.972297 15107 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:09.972765 15107 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:09.973199 15107 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:09.963122 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:09.983903 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:09.984535 11443 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:09.986838 11516 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:10.001549 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:10.001823 11519 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:10.005714 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.002511 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.008387 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:10.010288 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:09.982967 15092 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:10.027783 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:10.028376 11516 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:10.030161 15086 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:10.030591 15086 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:10.030934 15086 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:10.031916 11367 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:10.030967 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.015635 14852 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:10.015861 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.041450 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:10.041165 15114 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:16:10.045611 15113 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:10.046020 15114 consensus_queue.cc:1243] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:10.046161 15113 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.046548 14848 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.122s
I20260709 13:16:10.047631 14848 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:10.047976 14848 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:10.048568 15091 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.048739 14848 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.049304 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.049901 14848 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:10.050693 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.051034 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:10.051663 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.050045 12064 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:10.049104 15091 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.015117 11513 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:10.074054 12069 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:10.033622 14775 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:10.047107 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.079139 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.081192 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:10.081521 15097 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:10.081911 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.082283 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.082780 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.083254 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.083804 11518 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:10.048355 15113 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:10.091666 12065 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:16:10.043516 11409 connection.cc:256] RPC call timeout handler was delayed by -1.4149022399997193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
I20260709 13:16:10.091218 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.087196 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.092376 12066 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:10.090569 14811 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:10.097716 14811 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.098263 11361 pending_rounds.cc:85] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 6
I20260709 13:16:10.098654 12061 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:10.098977 11368 pending_rounds.cc:85] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 3
I20260709 13:16:10.099484 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.100636 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.100745 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.101475 11514 raft_consensus.cc:2368] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7.
I20260709 13:16:10.102243 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.085932 12064 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:10.103216 15114 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:10.103611 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:10.104262 12064 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:10.105664 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.097602 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:10.106840 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.107014 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:10.093156 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.107715 11518 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:10.013787 14633 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:16:10.113909 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.107654 12063 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:10.101720 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.103338 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.099256 11368 pending_rounds.cc:107] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:16:10.115846 11368 raft_consensus.cc:2889] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:10.114915 12061 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:10.098592 11361 pending_rounds.cc:107] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 8.7
W20260709 13:16:10.116298 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 69307.1us
W20260709 13:16:10.096813 11409 connection.cc:256] RPC call timeout handler was delayed by -1.2754022399997211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 13:16:10.111469 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:10.111673 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.097913 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:10.104967 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.036793 15041 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:16:10.023274 14855 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.118161 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:10.118995 12066 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:10.078351 14852 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:10.117311 11484 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.120985 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:10.117360 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.122536 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:10.116669 11361 raft_consensus.cc:2889] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:10.086474 14894 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.120393 14775 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:10.128156 12065 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:10.107324 15088 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.145581 14775 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.145951 15088 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.120605 14998 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:10.147727 11366 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:10.130584 14811 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:10.148749 11334 leader_election.cc:336] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.888s (SENT)
I20260709 13:16:10.149518 11514 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:10.155284 12063 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:10.127279 12069 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:10.133540 11443 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:10.137632 14783 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:10.137933 11511 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:10.123958 15092 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.139318 15090 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:10.140565 11444 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:10.141156 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.120728 15097 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:10.143754 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.144569 11408 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:10.148582 14811 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:10.171092 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:10.172320 11444 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:10.178246 12063 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:10.177681 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:10.192130 11517 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 7: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:10.185577 12032 leader_election.cc:400] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7.
I20260709 13:16:10.192764 12032 leader_election.cc:403] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:16:10.201530 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:10.203804 15148 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:10.204267 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 64910.1us
I20260709 13:16:10.205999 11512 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:10.206327 11516 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:10.198745 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:10.210440 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.212536 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:10.213202 11443 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:10.179884 15041 consensus_queue.cc:1243] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:10.219125 11333 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:10.219743 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56082.6us
I20260709 13:16:10.230178 12032 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.219672 15150 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:10.198323 14998 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:10.239466 14852 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.192214 14811 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:10.247437 11518 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:10.204154 15025 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:10.208114 15148 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.161439 14894 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.215466 14956 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:10.242041 15091 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:10.239552 11444 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:10.258642 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 107782us
W20260709 13:16:10.261171 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 116712us
I20260709 13:16:10.277585 11518 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:10.249845 15114 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:10.197683 15090 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:10.302896 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 154167us
I20260709 13:16:10.198431 15092 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.255642 15088 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:10.314170 11512 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:10.256201 14811 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:10.312527 12067 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
W20260709 13:16:10.321250 15134 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:16:10.336576 11517 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:10.337611 11518 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 13:16:10.321465 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 285027us
I20260709 13:16:10.321940 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:10.322728 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.322851 11363 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:10.322477 11362 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:10.278203 15091 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.348933 11370 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
W20260709 13:16:10.351999 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 121899us
I20260709 13:16:10.258464 15151 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:10.355492 14998 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.357612 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:10.358124 15098 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.359925 15113 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:16:10.361727 15144 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:16:10.361375 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.279121 15025 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:10.362457 14894 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:10.363215 11444 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:10.363595 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 50153.9us
I20260709 13:16:10.407477 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.365334 11512 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:10.260565 15148 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:10.366303 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:10.366636 11515 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:10.335305 15114 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:10.370911 15086 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.413117 14894 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:10.413923 15086 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.414505 15086 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:10.415433 15107 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.364461 15091 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.411568 14783 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.419950 14783 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.421942 14783 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:10.423874 15107 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.412207 14825 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:10.409900 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 48635.5us
I20260709 13:16:10.419140 15091 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:10.436195 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51809.6us
I20260709 13:16:10.429821 11510 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:10.439363 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.444902 11444 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:10.428851 14855 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:10.451220 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 193826us
I20260709 13:16:10.435717 14825 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.427536 15025 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.461799 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:10.384478 15098 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.426755 15087 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.456446 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.468398 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:10.445528 14894 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:16:10.454386 15091 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:10.447245 14633 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:16:10.498847 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:10.455217 14952 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:16:10.501260 11445 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:10.502677 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:10.503468 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:10.470052 14775 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:10.512216 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:10.512844 11441 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
W20260709 13:16:10.514873 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 107472us
I20260709 13:16:10.517664 11408 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.508927 14775 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.535137 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:10.516141 15092 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.556627 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:10.537964 14775 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:10.597904 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 76176.8us
W20260709 13:16:10.601260 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.601397 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 253059us
I20260709 13:16:10.532305 14846 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.167s
W20260709 13:16:10.613912 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55215.9us
I20260709 13:16:10.600696 15092 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:10.627847 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 135895us
W20260709 13:16:10.628193 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 71608.5us
I20260709 13:16:10.629031 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:10.629628 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.631220 11365 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:10.633286 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:10.633746 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.639858 11365 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:10.604522 14633 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:10.641607 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:10.627619 11408 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.647189 14983 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 13:16:10.649636 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:10.621639 15092 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:10.680627 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 67707.8us
I20260709 13:16:10.618551 14813 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:10.618300 15153 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:10.685312 14813 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:10.686461 15157 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:16:10.723276 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:10.687166 15098 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:10.692867 15168 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:10.695714 15163 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:10.696177 15169 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:10.684378 15092 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:10.696548 15165 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:10.697075 15160 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:16:10.699517 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:10.701304 15152 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:16:10.707705 15087 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:10.709070 14956 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7.
I20260709 13:16:10.650600 14894 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:16:10.711964 15153 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:10.760164 14848 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.710s	user 0.013s	sys 0.000s
I20260709 13:16:10.713361 14811 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.727975 11519 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:10.746584 11515 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:10.746762 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 142623us
W20260709 13:16:10.749163 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 121619us
I20260709 13:16:10.759678 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:10.760654 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 65715.2us
I20260709 13:16:10.719452 14633 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:16:10.781551 11514 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:16:10.700116 15148 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.743849 15163 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.742010 15168 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:10.777650 11513 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:10.747417 14825 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:10.797580 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 98126.6us
I20260709 13:16:10.781772 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:10.798326 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:10.783640 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 134086us
I20260709 13:16:10.772010 15114 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:10.779086 15025 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:10.796953 15087 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:16:10.800663 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 183511us
I20260709 13:16:10.795499 11513 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:10.791518 12033 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:10.803128 11512 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:10.804613 11511 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:10.811636 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:10.796310 15163 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:10.805127 15165 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:10.797367 15168 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:10.800444 15176 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:10.817677 11515 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:10.826107 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:10.826030 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44579.9us
I20260709 13:16:10.817178 14998 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:10.824240 11513 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.815995 14811 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.797269 15169 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:10.846091 15155 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:16:10.846511 15180 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:10.848042 15186 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:10.848570 15186 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:10.849511 15186 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.826761 15156 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:16:10.800758 14813 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.847965 15175 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 13:16:10.835767 15153 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.856077 14813 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:10.861653 11513 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:10.862020 11367 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:10.826963 15165 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:10.832991 15041 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:10.842025 15097 consensus_meta.cc:287] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.001s
I20260709 13:16:10.836119 15176 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:10.872435 15183 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:16:10.873600 15181 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.848474 15164 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:10.889068 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:10.868974 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:10.821125 15172 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.191s
I20260709 13:16:10.837287 14848 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 13:16:10.901803 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 59018.9us
I20260709 13:16:10.899130 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 847) took 1070 ms. Trace:
I20260709 13:16:10.899699 15164 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:10.892019 14633 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:16:10.891705 15181 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.908413 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.903894 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:10.882472 15097 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:10.900233 15092 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.883471 15041 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.902417 11362 rpcz_store.cc:276] 0709 13:16:09.831448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.831576 (+   128us) service_pool.cc:224] Handling call
0709 13:16:09.831873 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.348484 (+516611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.898892 (+550408us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b5400166054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:10.899072 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780534656}
W20260709 13:16:10.904661 11483 connection.cc:256] RPC call timeout handler was delayed by -1.2114952800000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
W20260709 13:16:10.905174 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63967.1us
W20260709 13:16:10.911556 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 77987.5us
I20260709 13:16:10.913748 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.914324 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:10.904148 15025 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:10.912737 15180 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.917132 15181 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:10.916970 15180 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.920254 11364 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:16:10.921068 15173 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:10.921242 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.922514 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:10.922984 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:10.924002 15174 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:10.924435 11368 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:10.924973 15061 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:10.925338 15061 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:10.924456 15174 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.927245 15174 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:10.931825 15165 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:10.925349 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:10.933099 11368 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:10.936650 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:10.939594 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 58323.8us
I20260709 13:16:10.942591 11364 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:10.944307 12034 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:10.951200 15086 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:10.959108 14998 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:10.959545 15041 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.959965 15171 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:10.960721 15086 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:10.960507 15171 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.961308 15086 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.962742 15171 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.960402 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:10.934437 15173 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:10.919243 15180 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.913411 15164 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:10.965808 11444 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:16:10.964768 14852 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.015s	sys 0.009s
I20260709 13:16:10.971204 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:10.973560 11443 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:10.977942 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:10.978770 11445 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:10.959815 15182 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:10.959108 15169 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:10.975021 14852 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.016s	sys 0.009s
I20260709 13:16:10.963331 15086 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.975028 15192 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.000283 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.981316 15148 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:10.994232 15183 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:11.003911 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:11.002638 15164 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:10.959784 14998 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:11.003738 15183 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:10.972662 15175 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:16:10.983696 14848 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:16:10.983934 15169 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:11.007459 15183 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:11.045759 11513 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:16:11.046854 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:11.050307 11513 pending_rounds.cc:85] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 4
I20260709 13:16:11.051730 11513 pending_rounds.cc:107] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:16:11.052047 11513 raft_consensus.cc:2889] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:11.045146 15190 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:11.061239 15190 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:11.015478 15182 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:16:11.062234 15190 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:11.059993 15097 raft_consensus.cc:2764] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:16:11.060376 15184 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:16:11.045238 15186 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:16:10.984055 15176 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:11.053784 11364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:11.070822 11364 raft_consensus.cc:606] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
W20260709 13:16:11.065394 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 49500.6us
I20260709 13:16:11.066092 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:11.061254 15151 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.000s
I20260709 13:16:11.046661 15169 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:11.063481 15196 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:11.075230 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:11.076339 11513 raft_consensus.cc:1217] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Deduplicated request from leader. Original: 4.6->[6.7-7.8]   Dedup: 6.7->[7.8-7.8]
I20260709 13:16:11.045090 15153 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:16:11.077708 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:11.079124 11513 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:16:11.063318 15182 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:11.080406 11366 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:11.081967 11514 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:11.083809 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:11.094863 15195 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:11.077410 15151 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:11.078143 15196 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.081658 15182 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:11.085963 15148 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:11.104341 11369 raft_consensus.cc:1217] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[8.8-10.9]   Dedup: 10.9->[]
I20260709 13:16:11.011754 15192 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:11.101222 15041 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:16:11.110641 15182 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:11.106693 15196 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:11.012267 15173 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:11.116650 11369 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:16:11.117163 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:12.257280 15209 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.713970 15209 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:12.952776 15214 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.071254 15215 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.094727 15216 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.717077 15214 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.717464 15216 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.124773 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:13.719931 11440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2448) took 3209 ms (client timeout 2152 ms). Trace:
W20260709 13:16:13.720278 11440 rpcz_store.cc:269] 0709 13:16:10.510381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.510582 (+   201us) service_pool.cc:224] Handling call
0709 13:16:13.124571 (+2613989us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:16:13.719819 (+595248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20172032}
I20260709 13:16:13.150486 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 801) took 3169 ms. Trace:
I20260709 13:16:13.170068 15220 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:13.173031 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.176081 15221 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:13.180176 15222 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:13.717120 15215 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:13.192684 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.721292 11510 rpcz_store.cc:276] 0709 13:16:09.981376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.981524 (+   148us) service_pool.cc:224] Handling call
0709 13:16:10.002495 (+ 20971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.443570 (+441075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.150175 (+2706605us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54000ea954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:13.150387 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979926912}
W20260709 13:16:11.561442 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 352949us
W20260709 13:16:13.120357 12067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 705) took 3069 ms (client timeout 1717 ms). Trace:
W20260709 13:16:13.727329 12067 rpcz_store.cc:269] 0709 13:16:10.051106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.087775 (+ 36669us) service_pool.cc:224] Handling call
0709 13:16:13.120060 (+3032285us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54005d6954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:13.120222 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444917632}
I20260709 13:16:13.727356 15211 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:13.728657 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:13.099192 12032 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:13.090272 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:13.187309 15223 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:13.091207 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:11.209280 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:11.207509 12062 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:13.738210 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 959) took 2532 ms. Trace:
I20260709 13:16:13.738516 12062 rpcz_store.cc:276] 0709 13:16:11.205679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.206738 (+  1059us) service_pool.cc:224] Handling call
0709 13:16:11.207057 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:13.737935 (+2530878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.738125 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:11.208588 12065 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:13.741225 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 960) took 2535 ms. Trace:
I20260709 13:16:13.741663 12065 rpcz_store.cc:276] 0709 13:16:11.205969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.207944 (+  1975us) service_pool.cc:224] Handling call
0709 13:16:11.208240 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:13.740872 (+2532632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.741156 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:11.210381 12069 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:11.206261 12066 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:13.085022 14638 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:16:11.158411 14775 consensus_meta.cc:287] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.001s
I20260709 13:16:13.887629 14775 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.888749 11439 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:13.889856 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 724) took 2933 ms. Trace:
I20260709 13:16:13.890158 11439 rpcz_store.cc:276] 0709 13:16:10.956227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.956924 (+   697us) service_pool.cc:224] Handling call
0709 13:16:10.972268 (+ 15344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.889089 (+2916821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.889583 (+   494us) spinlock_profiling.cc:243] Waited 2.92 s on lock 0x7b54002080d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:13.889773 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6706799488}
I20260709 13:16:13.891597 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:13.892191 11439 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:11.203334 14848 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:13.899827 15249 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.900384 15249 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.900950 11439 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:13.902489 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:13.902417 15249 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:11.105289 14983 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:16:13.931170 14983 consensus_queue.cc:1243] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:13.931674 14983 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.932094 14983 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.935225 15215 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:11.163940 14998 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.037614 15061 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.966168 15061 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:13.967207 14983 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.968047 11514 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:13.968436 11514 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:13.969203 11514 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.970086 11514 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:13.169210 15087 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:11.167915 15093 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:14.009893 15093 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:14.010315 15162 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.010808 15093 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.012320 15093 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:12.169248 15113 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:13.205338 15147 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:16:13.168522 15155 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:14.036986 15155 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:14.037886 15155 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.038863 15155 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:11.182152 15163 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.041564 11518 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:14.042517 15163 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:11.168962 15168 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.045264 15168 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:11.188802 15169 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.046078 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.047271 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.047891 15169 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:11.101446 15172 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:14.049082 15172 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:14.049573 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.049942 15172 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.051026 15172 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.051563 12068 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:14.053887 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.054980 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:11.183204 15174 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.057621 15174 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:11.173801 15180 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:11.077262 15181 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:16:11.187039 15183 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.070436 15183 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:11.682646 15187 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:14.071785 15187 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:11.159617 15192 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:11.169762 15196 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.078697 15196 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:11.160549 15197 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:16:11.721534 15200 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:14.081761 15200 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.083945 15200 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:13.101311 15206 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.939s	user 0.018s	sys 0.010s
W20260709 13:16:13.707753 11408 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.193s (SENT) [suppressed 28 similar messages]
I20260709 13:16:13.716454 15209 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:13.721982 15222 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.087144 15222 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:13.721918 15221 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.105623 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.105875 15221 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.721657 15220 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:13.726723 15213 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:14.107374 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:13.734221 15225 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:13.734833 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.091984 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:14.109654 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1158) took 1018 ms. Trace:
I20260709 13:16:14.109987 12063 rpcz_store.cc:276] 0709 13:16:13.091112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.091693 (+   581us) service_pool.cc:224] Handling call
0709 13:16:14.109594 (+1017901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:14.110688 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:13.735705 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:14.114322 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1162) took 1021 ms. Trace:
I20260709 13:16:14.114110 15225 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.114691 12067 rpcz_store.cc:276] 0709 13:16:13.092649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.729746 (+637097us) service_pool.cc:224] Handling call
0709 13:16:14.114236 (+384490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:14.115826 12067 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:14.116415 15225 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.743613 15214 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:13.745491 15223 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.108086 11445 raft_consensus.cc:2368] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8.
I20260709 13:16:14.118136 11262 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 4 to 12, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 12 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:13.746683 15226 raft_consensus.cc:2764] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 13:16:13.747575 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 2.63033e+06us
I20260709 13:16:14.119660 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.120476 11445 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:14.120095 11262 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 6 to 7, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:14.121160 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.121711 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.121589 11262 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 6, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:14.122371 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 414625us
W20260709 13:16:13.756798 15191 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 2.792s	user 0.021s	sys 0.021s
I20260709 13:16:14.124306 15221 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:13.750573 15229 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.766698 14894 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.768637 15228 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:14.119652 15223 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:13.770088 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 670949us
I20260709 13:16:14.133843 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.135417 15213 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:13.771881 15227 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:14.136117 15213 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:14.136258 15227 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.137112 15213 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:14.137455 15213 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:14.138413 15227 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:14.138232 15213 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.776037 15233 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.133186 15228 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.139688 15233 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.141014 15228 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.777597 15235 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.778780 15156 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.780452 14855 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.141769 15233 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.141769 15235 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.142251 14855 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.144268 14855 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:13.782258 14635 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.144692 15235 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.144882 14635 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.146883 14635 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:13.787142 15217 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:14.147658 15217 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.148762 15213 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.149597 15217 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:13.792627 14783 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:14.150254 14783 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.152716 14783 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.791083 15238 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:14.153468 15238 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.776139 15232 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:13.801661 15114 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.155733 15232 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.155901 15114 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:14.156878 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.157840 15232 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.803979 15239 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.158728 15239 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.159374 15114 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.160811 15239 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:13.805107 15234 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:13.807276 11519 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:13.809484 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 818) took 3359 ms. Trace:
I20260709 13:16:14.161279 15200 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:13.809700 11516 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:16:13.813489 11511 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:14.161654 15214 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.161867 15200 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.162367 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.162559 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.163581 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.163725 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:14.163932 15200 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:13.819031 15182 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:13.824177 15092 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.167160 15238 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.825847 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2301) took 2915 ms. Trace:
I20260709 13:16:13.830219 15216 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:14.170759 11266 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39444 (request call id 352) took 14124 ms (client timeout 15000 ms). Trace:
I20260709 13:16:13.832984 14846 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:13.833711 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:13.832994 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.171283 14846 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:14.171666 11444 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:14.171979 15217 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:14.172322 14846 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.172768 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:13.837929 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:13.842309 15240 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:13.847080 15244 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:14.173588 15240 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:13.852434 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.174643 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.175005 11444 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:16:14.175168 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.175625 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.175830 15240 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.176167 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:13.852452 15248 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.176566 11444 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:14.176566 11335 connection.cc:256] RPC call timeout handler was delayed by -1.1966339609998613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 13:16:14.176832 15248 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.173070 15217 consensus_queue.cc:1059] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:16:13.874780 11443 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:13.882611 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 962) took 2676 ms. Trace:
I20260709 13:16:14.178120 12069 rpcz_store.cc:276] 0709 13:16:11.206572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.209765 (+  3193us) service_pool.cc:224] Handling call
0709 13:16:11.210142 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.882268 (+2672126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.882536 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:13.883986 15237 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:14.177350 11335 connection.cc:256] RPC call timeout handler was delayed by -1.051976606999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
I20260709 13:16:13.883281 15231 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:14.179359 11335 connection.cc:256] RPC call timeout handler was delayed by -1.0288045949998832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
I20260709 13:16:14.180222 15225 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:14.180923 12064 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:14.180754 15225 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.179168 15237 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.873420 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 867) took 2966 ms. Trace:
I20260709 13:16:14.182794 15172 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:13.903115 11439 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:13.908826 15247 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:13.885560 14638 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:13.915565 15249 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.914183 15088 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.184381 14638 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.184809 15249 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.184928 15088 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:13.910692 15170 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:14.185494 11439 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:14.185891 15170 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.186751 15249 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.187449 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.188130 15170 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.188675 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:14.189188 15225 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.174216 15244 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:13.922097 14775 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.190016 15088 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:13.923288 15176 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.008s	sys 0.004s
W20260709 13:16:14.190572 15176 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.010s	sys 0.004s
I20260709 13:16:13.926481 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:14.191182 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.191743 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1156) took 1101 ms. Trace:
I20260709 13:16:13.945925 14952 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:14.184465 15200 raft_consensus.cc:3055] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:14.192442 14952 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:14.192847 15200 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:13.950915 15250 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.961385 15258 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.193739 15200 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.194286 15258 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.189509 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:14.192157 15237 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.171263 11369 rpcz_store.cc:276] 0709 13:16:10.910198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.923559 (+ 13361us) service_pool.cc:224] Handling call
0709 13:16:11.072278 (+148719us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:13.711391 (+2639113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.825650 (+114259us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000c7450. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:13.825772 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321818368}
I20260709 13:16:14.195025 15200 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.196785 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.197183 15258 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:13.980458 15087 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:13.989912 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2141) took 3908 ms. Trace:
W20260709 13:16:13.983253 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 2.88111e+06us
I20260709 13:16:14.198779 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:14.004681 11514 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:14.195564 11367 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.200139 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.198257 15087 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.201215 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.006278 15154 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:16:14.193722 14952 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.202523 15154 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:14.202425 15231 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:14.202907 15154 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:14.010968 15162 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.203801 15154 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.204923 15154 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:14.205299 15162 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:11.178001 15148 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:14.017668 11365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:14.207314 11365 raft_consensus.cc:606] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.023305 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.183033 11370 rpcz_store.cc:276] 0709 13:16:10.906822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.907761 (+   939us) service_pool.cc:224] Handling call
0709 13:16:10.908073 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:11.049516 (+141443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:11.049545 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:11.053856 (+  4311us) log.cc:844] Serialized 26 byte log entry
0709 13:16:11.082677 (+ 28821us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:11.082711 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.163819 (+2081108us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b1800446d40. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:13.873099 (+709280us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b54002dcb54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:13.873153 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:13.873195 (+    42us) raft_consensus.cc:1685] finished
0709 13:16:13.873207 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:13.873358 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124639,"raft.queue_time_us":104440,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":86,"spinlock_wait_cycles":85169920,"thread_start_us":17583,"threads_started":1,"wal-append.queue_time_us":17957,"wal-append.run_cpu_time_us":12131,"wal-append.run_wall_time_us":1001881}
I20260709 13:16:14.208577 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.209493 11370 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:14.209677 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.025197 11363 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:14.035363 15246 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.211782 15246 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.212651 12033 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.213387 15231 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:14.025774 15113 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.043124 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2148) took 3321 ms. Trace:
I20260709 13:16:14.060051 15180 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:14.062973 15190 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.214879 11518 rpcz_store.cc:276] 0709 13:16:10.722253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.722993 (+   740us) service_pool.cc:224] Handling call
0709 13:16:10.744058 (+ 21065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:14.041921 (+3297863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.042919 (+   998us) spinlock_profiling.cc:243] Waited 3.3 s on lock 0x7b540014c8d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:14.043071 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7583222144}
I20260709 13:16:14.075496 15187 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:14.216035 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.087163 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 725) took 3118 ms. Trace:
I20260709 13:16:14.179008 11443 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 13:16:14.097149 15181 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:14.217108 15181 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:14.207684 11365 raft_consensus.cc:614] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:14.198159 11517 rpcz_store.cc:276] 0709 13:16:10.081417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.211181 (+129764us) service_pool.cc:224] Handling call
0709 13:16:10.336386 (+125205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.337052 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.989661 (+3652609us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000e8150. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:13.989836 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270903552}
I20260709 13:16:14.218014 15181 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.102268 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:14.219182 15113 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.219515 11510 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:14.219790 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.220934 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.103427 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.221206 15148 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.096175 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:14.108963 11334 connection.cc:256] RPC call timeout handler was delayed by -2.0903500560000339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
I20260709 13:16:14.111238 11440 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:16:14.222172 11334 connection.cc:256] RPC call timeout handler was delayed by -1.3688153640000564s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 13:16:14.108042 15220 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.222743 15246 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:14.222954 15170 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.201452 15087 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:14.223511 15170 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.222715 11334 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.224650 15180 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.225697 15180 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:14.226619 15093 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:13.736263 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.227566 15192 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:13.763712 15173 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.228209 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.227142 15093 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.221611 15235 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.230562 15093 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.130854 14894 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.123795 11262 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 5 to 7, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:14.231685 15192 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:14.231904 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.231074 15093 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:14.232394 11262 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 5 to 7, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:14.132148 11445 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:14.234769 15192 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.236901 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.228849 15176 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:14.185665 14638 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.238962 14845 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:14.129347 15229 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:13.820253 15245 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:14.164809 15234 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.162293 15214 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.141855 15156 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.131304 12034 leader_election.cc:400] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8.
I20260709 13:16:14.165318 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.164147 11519 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:13.831626 15090 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:14.176337 15240 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:14.164407 11511 raft_consensus.cc:2368] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8.
I20260709 13:16:14.178689 11444 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:16:14.178992 15248 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:13.883925 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 958) took 2678 ms. Trace:
I20260709 13:16:14.183876 15247 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:13.913987 11484 connection.cc:256] RPC call timeout handler was delayed by -1.9483874689999539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 13:16:13.904047 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:14.189746 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:13.920847 14848 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:13.928435 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1666) took 2951 ms. Trace:
I20260709 13:16:14.198549 15250 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.206416 15239 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:14.206708 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.005535 14983 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.132473 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.216795 11518 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:14.217146 11368 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:14.218982 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.227821 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1160) took 1135 ms. Trace:
I20260709 13:16:14.233052 14894 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.177399 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.161705 11512 rpcz_store.cc:276] 0709 13:16:10.450134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.456085 (+  5951us) service_pool.cc:224] Handling call
0709 13:16:10.456455 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.814832 (+358377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.809268 (+2994436us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b5400197dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:13.809434 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796848512}
I20260709 13:16:14.235275 15244 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.210537 11370 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:14.201988 11514 raft_consensus.cc:2410] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:14.343966 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.345261 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.345713 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.345903 11511 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:14.346117 14848 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.346908 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.348071 12033 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.349013 15247 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.245227 15170 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:14.262322 15233 consensus_queue.cc:1175] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34a42aa6aa0a4778a532c6433d70b981" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 9 } }
W20260709 13:16:14.350906 11484 connection.cc:256] RPC call timeout handler was delayed by -1.922287469000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:16:14.352500 15229 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:14.351979 14983 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.251204 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.353227 12068 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:14.353518 11361 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:16:14.354912 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.355258 14983 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.355494 11361 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:16:14.243995 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.219235 15181 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.359113 12066 rpcz_store.cc:276] 0709 13:16:11.205338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.205625 (+   287us) service_pool.cc:224] Handling call
0709 13:16:11.206024 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:13.883696 (+2677672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.883876 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:14.259236 14635 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:16:14.173429 14846 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.300475 11262 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:14.261072 11365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "857937fa61dc400a81345944f0a14511"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:14.364020 11365 raft_consensus.cc:606] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.363502 11262 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 4 to 8, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:14.324139 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 92290.7us
I20260709 13:16:14.203138 14952 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:14.304744 15196 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.321604 15235 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:14.366354 15196 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.368072 12062 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:14.368710 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1163) took 1275 ms. Trace:
I20260709 13:16:14.368877 11437 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:16:14.313457 11440 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:14.368588 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 116744us
I20260709 13:16:14.192126 12064 rpcz_store.cc:276] 0709 13:16:13.090108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.090214 (+   106us) service_pool.cc:224] Handling call
0709 13:16:13.883686 (+793472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:14.181265 (+297579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.190890 (+  9625us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54001faed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:14.191554 (+   664us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b54001faed0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:14.191664 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506815744}
I20260709 13:16:14.343242 12034 leader_election.cc:403] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:16:14.239341 11510 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:16:14.371551 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 240371us
I20260709 13:16:14.259660 15253 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.342957 15239 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:14.372586 15253 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.349750 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.374615 11370 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:14.342846 15245 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.375756 15239 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.376587 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.377163 11518 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:14.377529 15245 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.344264 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.378109 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.348209 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.375169 14894 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:14.380038 14894 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.380232 15253 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.346585 15240 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.454564 15240 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:14.381084 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.381845 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 258684us
I20260709 13:16:14.381783 12034 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.456120 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:14.456352 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 74630.3us
I20260709 13:16:14.456436 15200 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:14.456813 15170 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:14.457244 15170 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:14.457577 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.456861 15258 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:14.458036 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:14.458664 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.354717 11438 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:14.457966 15170 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.350152 15214 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.380542 14894 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.344101 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.459883 15170 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.462220 15229 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.462949 15229 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:14.447065 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.464010 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.458398 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.465126 11370 raft_consensus.cc:2410] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:16:14.468034 12032 leader_election.cc:400] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8.
I20260709 13:16:14.256677 11436 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:14.383986 12060 rpcz_store.cc:276] 0709 13:16:13.092141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.735218 (+643077us) service_pool.cc:224] Handling call
0709 13:16:14.227765 (+492547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704704}
I20260709 13:16:14.216661 11441 rpcz_store.cc:276] 0709 13:16:10.969181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.969304 (+   123us) service_pool.cc:224] Handling call
0709 13:16:10.981066 (+ 11762us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:10.996296 (+ 15230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:10.996339 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:11.008224 (+ 11885us) log.cc:844] Serialized 26 byte log entry
0709 13:16:11.123862 (+115638us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:13.100724 (+1976862us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:13.114500 (+ 13776us) log.cc:844] Serialized 14 byte log entry
0709 13:16:13.128378 (+ 13878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.128468 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:14.086887 (+958419us) raft_consensus.cc:1685] finished
0709 13:16:14.086988 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:14.087110 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1990759,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":15894,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":17221,"spinlock_wait_cycles":4448000,"thread_start_us":64399,"threads_started":3,"wal-append.queue_time_us":111943}
I20260709 13:16:14.384215 15234 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.378537 11514 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:14.386621 12062 rpcz_store.cc:276] 0709 13:16:13.092870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.926167 (+833297us) service_pool.cc:224] Handling call
0709 13:16:14.368655 (+442488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:14.387003 15196 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.374881 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.366398 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:14.351326 11484 connection.cc:256] RPC call timeout handler was delayed by -1.7206874690000404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260709 13:16:14.471161 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 99922.6us
W20260709 13:16:14.471437 11484 connection.cc:256] RPC call timeout handler was delayed by -1.6138970409999729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
I20260709 13:16:14.159476 15221 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:14.347376 11515 rpcz_store.cc:276] 0709 13:16:10.977124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.977383 (+   259us) service_pool.cc:224] Handling call
0709 13:16:10.992141 (+ 14758us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:11.007757 (+ 15616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:11.007787 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:11.050024 (+ 42237us) log.cc:844] Serialized 26 byte log entry
0709 13:16:11.128693 (+ 78669us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:11.128737 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.128816 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:13.928167 (+2799351us) raft_consensus.cc:1685] finished
0709 13:16:13.928192 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:13.928365 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1471,"raft.queue_time_us":14008,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":33222,"thread_start_us":64913,"threads_started":1,"wal-append.queue_time_us":102361,"wal-append.run_cpu_time_us":1594,"wal-append.run_wall_time_us":1024973}
I20260709 13:16:14.473029 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.473634 11515 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:14.474458 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.475013 11515 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:14.346774 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.475696 11512 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
I20260709 13:16:14.476562 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.477133 11512 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:14.477960 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.478447 11512 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:14.478793 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.386518 11366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a38969de5c824718aa81b9622624bc51"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:14.480119 11366 raft_consensus.cc:606] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.347198 14848 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:14.393313 11262 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 5 to 4, leader changed from <none> to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:14.481745 11262 catalog_manager.cc:5697] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 7 to 8, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 8 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:14.483290 11262 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:14.484673 11262 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 5 to 7, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:14.346334 15156 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:14.415221 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.378723 11518 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:14.479156 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.370003 15250 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.446676 15245 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:14.444058 12065 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:14.447178 15241 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.487687 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1159) took 1395 ms. Trace:
I20260709 13:16:14.487728 15241 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.487159 11512 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:14.488219 12065 rpcz_store.cc:276] 0709 13:16:13.091885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.899188 (+807303us) service_pool.cc:224] Handling call
0709 13:16:14.487628 (+588440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:14.488799 11514 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:16:14.489944 11514 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:14.487169 11518 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:14.221285 11367 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:14.451826 11540 debug-util.cc:398] Leaking SignalData structure 0x7b08001ecb00 after lost signal to thread 10260
I20260709 13:16:14.459789 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.492074 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:14.492447 11540 debug-util.cc:398] Leaking SignalData structure 0x7b080018c320 after lost signal to thread 11302
I20260709 13:16:14.492755 11519 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:14.493021 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.460211 15258 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:14.493726 11540 debug-util.cc:398] Leaking SignalData structure 0x7b08001ba5c0 after lost signal to thread 11394
I20260709 13:16:14.493799 15258 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:14.494186 11540 debug-util.cc:398] Leaking SignalData structure 0x7b08000bed20 after lost signal to thread 11395
I20260709 13:16:14.494278 15258 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:14.495478 11540 debug-util.cc:398] Leaking SignalData structure 0x7b0800370040 after lost signal to thread 11469
W20260709 13:16:14.495726 11540 debug-util.cc:398] Leaking SignalData structure 0x7b08003acd40 after lost signal to thread 11470
W20260709 13:16:14.496721 11540 debug-util.cc:398] Leaking SignalData structure 0x7b080031cb80 after lost signal to thread 11543
W20260709 13:16:14.496969 11540 debug-util.cc:398] Leaking SignalData structure 0x7b080040a080 after lost signal to thread 11544
I20260709 13:16:14.497843 12061 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:14.488134 11515 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:14.486023 11262 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 6 to 7, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 7 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:14.373648 14638 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:14.498005 11540 debug-util.cc:398] Leaking SignalData structure 0x7b0800192540 after lost signal to thread 12093
I20260709 13:16:14.499374 11368 raft_consensus.cc:2410] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:16:14.499532 11540 debug-util.cc:398] Leaking SignalData structure 0x7b080034d840 after lost signal to thread 12094
I20260709 13:16:14.486951 11517 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:14.499694 15220 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:14.500521 15250 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:14.501682 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:14.499965 11262 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 7 to 8, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 8 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.485209 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:14.472048 11484 connection.cc:256] RPC call timeout handler was delayed by -1.483087469000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 13:16:14.471437 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 96607.2us
W20260709 13:16:14.390946 11266 rpcz_store.cc:269] 0709 13:16:00.046491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.046857 (+   366us) service_pool.cc:224] Handling call
0709 13:16:00.539127 (+492270us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b54001897d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc968090 00007f35bca06f3a 00007f35c27bd1cc 00007f35c28bb37e 00007f35c0d7c2a4 00007f35c0d7c232 00007f35c0d7c161 00007f35c0d7c0dc 00007f35c0d7b392 00007f35bad3948c 00007f35bad388c6
0709 13:16:14.170583 (+13631456us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:09.421858 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:09.422694 (+   836us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:09.453227 (+ 30533us) write_op.cc:435] Acquired schema lock
0709 13:16:09.456159 (+  2932us) tablet.cc:662] Decoding operations
0709 13:16:09.636503 (+180344us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:09.636590 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:09.636615 (+    25us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:16:09.639036 (+  2421us) tablet.cc:701] Row locks acquired
0709 13:16:09.639086 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:16:09.639270 (+   184us) write_op.cc:270] Start()
0709 13:16:09.645801 (+  6531us) write_op.cc:276] Timestamp: P: 1783602969639205 usec, L: 0
0709 13:16:09.647379 (+  1578us) op_driver.cc:348] REPLICATION: starting
0709 13:16:09.727033 (+ 79654us) log.cc:844] Serialized 9413 byte log entry
0709 13:16:10.203718 (+476685us) op_driver.cc:464] REPLICATION: finished
0709 13:16:10.561292 (+357574us) write_op.cc:301] APPLY: starting
0709 13:16:10.594243 (+ 32951us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:10.647254 (+ 53011us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:10.647444 (+   190us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:10.688055 (+ 40611us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:10.688511 (+   456us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 13:16:10.688562 (+    51us) write_op.cc:312] APPLY: finished
0709 13:16:10.689065 (+   503us) log.cc:844] Serialized 164 byte log entry
0709 13:16:10.689250 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:10.689552 (+   302us) write_op.cc:454] Released schema lock
0709 13:16:10.689641 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:10.690099 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:10.690128 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":52763,"reported_tablets":46,"spinlock_wait_cycles":11673344,"child_traces":[["op",{"apply.queue_time_us":312990,"apply.run_cpu_time_us":6791,"apply.run_wall_time_us":156308,"num_ops":24,"prepare.queue_time_us":87627,"prepare.run_cpu_time_us":36176,"prepare.run_wall_time_us":360229,"raft.queue_time_us":245559,"raft.run_cpu_time_us":11086,"raft.run_wall_time_us":254869,"replication_time_us":556184,"thread_start_us":542136,"threads_started":4,"wal-append.queue_time_us":185317,"wal-append.run_cpu_time_us":38687,"wal-append.run_wall_time_us":1873725}]]}
I20260709 13:16:14.506294 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.512410 11437 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:14.511638 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:14.514900 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.515106 11438 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:14.513392 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.516171 12033 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:14.452478 15239 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:14.517191 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45135.4us
W20260709 13:16:14.517678 12061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 961) took 3311 ms (client timeout 1615 ms). Trace:
I20260709 13:16:14.518486 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:14.518044 12061 rpcz_store.cc:269] 0709 13:16:11.206261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.209003 (+  2742us) service_pool.cc:224] Handling call
0709 13:16:14.517520 (+3308517us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b54000587d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:14.517615 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1734996736}
I20260709 13:16:14.519059 11518 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:14.519891 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.520990 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.523022 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.523655 11516 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:14.519757 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.525722 15215 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:14.526948 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.525163 11262 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 6 to 8, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:14.525197 11438 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:14.531198 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.533267 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:14.535465 11440 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:14.536360 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.470235 15214 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:14.539762 11262 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 4 to 7, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 7 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:14.540553 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.541131 11437 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
W20260709 13:16:14.538514 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 172169us
I20260709 13:16:14.544142 12062 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:16:14.544523 12062 pending_rounds.cc:85] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 6
I20260709 13:16:14.544730 12062 pending_rounds.cc:107] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:16:14.544950 12062 raft_consensus.cc:2889] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:14.545168 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.546077 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.546973 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.547688 12066 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:14.549774 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.549996 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.551146 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:14.551996 11512 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:14.552307 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:14.552856 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 74145.1us
I20260709 13:16:14.521791 14952 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:14.554746 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.546828 15214 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.557617 15214 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:14.555294 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.561299 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.537866 14775 raft_consensus.cc:2764] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:16:14.566140 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.540735 15234 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:14.568051 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:14.568509 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49553.1us
I20260709 13:16:14.568733 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.570182 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.546732 15216 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:14.574445 12062 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:14.576148 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:14.576930 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 43734us
I20260709 13:16:14.577543 11367 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:14.577587 11333 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.577466 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.552484 15113 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:14.549297 15215 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:14.582508 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:14.578662 11483 connection.cc:256] RPC call timeout handler was delayed by -2.1579213170000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:16:14.578666 11368 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:16:14.583712 11483 connection.cc:256] RPC call timeout handler was delayed by -2.1300213170000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
I20260709 13:16:14.576323 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.584026 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:14.584043 11483 connection.cc:256] RPC call timeout handler was delayed by -2.1237213169999904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 13:16:14.584718 11483 connection.cc:256] RPC call timeout handler was delayed by -2.063421316999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
W20260709 13:16:14.584939 11483 connection.cc:256] RPC call timeout handler was delayed by -1.9950213170000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
W20260709 13:16:14.585197 11483 connection.cc:256] RPC call timeout handler was delayed by -1.9563213169999472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 13:16:14.585683 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:14.586018 11483 connection.cc:256] RPC call timeout handler was delayed by -1.9131213169998773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 13:16:14.586352 11483 connection.cc:256] RPC call timeout handler was delayed by -1.8501213170000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 13:16:14.586659 11483 connection.cc:256] RPC call timeout handler was delayed by -1.7772213169998849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
I20260709 13:16:14.554862 14952 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.588109 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.558039 15230 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.602391 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.603055 11436 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 4.
W20260709 13:16:14.586957 11483 connection.cc:256] RPC call timeout handler was delayed by -1.7772213169998849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
I20260709 13:16:14.588828 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:14.591711 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:14.558156 15240 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:14.586452 11368 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
W20260709 13:16:14.604496 11483 connection.cc:256] RPC call timeout handler was delayed by -1.7187213170000177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
I20260709 13:16:14.605149 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.605182 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.604236 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.608886 11370 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:14.594624 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.609977 11361 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:14.607048 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.605561 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.603832 12060 raft_consensus.cc:2376] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8. Re-sending same reply.
I20260709 13:16:14.613667 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.571053 15233 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:14.616106 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:14.612989 12069 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:16:14.615516 11409 connection.cc:256] RPC call timeout handler was delayed by -3.1631823920001807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 13:16:14.616986 11409 connection.cc:256] RPC call timeout handler was delayed by -2.9768823920001068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
W20260709 13:16:14.617271 11409 connection.cc:256] RPC call timeout handler was delayed by -2.8184823920000781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260709 13:16:14.617550 11409 connection.cc:256] RPC call timeout handler was delayed by -2.7509823920001963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
W20260709 13:16:14.617813 11409 connection.cc:256] RPC call timeout handler was delayed by -2.2541823920000752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2546s for a response
I20260709 13:16:14.618443 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:14.619539 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.579259 15150 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.622982 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.619438 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.587111 15113 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.557993 15235 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:16:14.644256 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68001us
I20260709 13:16:14.555163 15092 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:16:14.582060 15190 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:14.625895 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.602113 14952 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.649037 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.612996 15240 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.628377 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.579675 15181 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
W20260709 13:16:14.666200 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88696.5us
I20260709 13:16:14.667182 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:14.666041 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:14.645519 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 116542us
I20260709 13:16:14.657896 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:14.660177 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.664323 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.664263 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:14.666304 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47997.3us
I20260709 13:16:14.670555 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.675187 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:14.641853 15113 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.620268 15233 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.627424 15183 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:14.625016 15234 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.677439 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:14.668622 12060 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:14.649065 15217 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:14.713672 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:14.705509 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.662333 15190 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.725427 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.726301 11443 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:16:14.727643 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:14.728274 11365 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:14.668818 14846 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:14.644619 15182 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:16:14.668716 15088 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:14.659574 14635 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:16:14.731285 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.669871 15170 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:14.732034 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 64931.9us
I20260709 13:16:14.746734 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:14.732340 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 74531.2us
I20260709 13:16:14.673259 15093 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:16:14.732542 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 57929.4us
I20260709 13:16:14.752336 11361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:14.641494 15150 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.673106 15246 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:16:14.712307 15232 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:14.722764 15217 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.752166 11443 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:14.774598 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.775468 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.752975 11361 raft_consensus.cc:606] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.775741 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.776147 11361 raft_consensus.cc:614] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:14.776366 11445 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 4.
I20260709 13:16:14.667631 15240 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:14.777186 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.754513 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.730921 14783 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:14.724545 15087 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:14.732908 15190 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.778265 15180 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:14.763024 15170 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.778681 15180 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.779153 15180 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.780763 14783 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.781055 15170 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.781152 14783 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.784121 11334 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.784924 15258 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.785926 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.786643 15253 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:14.787242 15253 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:14.787688 11440 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:14.754905 15093 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.789119 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.789461 15093 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:14.790956 15241 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.791247 15217 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.764390 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:14.794147 11333 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:14.792071 14783 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.795082 14846 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:14.796219 14846 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.796641 14846 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:14.741113 15248 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:14.798022 15248 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:14.798470 15248 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.751688 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:14.773417 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.800505 11439 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:14.789099 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:14.739226 15249 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:14.795575 15247 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:14.722309 15233 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:14.745843 15148 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.745072 15221 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:14.812572 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.787139 15258 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.823858 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:14.826037 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 150915us
I20260709 13:16:14.828152 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:14.829682 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:14.834034 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 120419us
I20260709 13:16:14.835326 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:14.839622 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.812825 14855 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:14.741272 15225 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:16:14.898573 14846 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.917985 14846 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.731640 15215 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.921190 15217 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.889134 15180 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:14.899921 15279 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260709 13:16:14.907223 15276 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:16:14.925923 15180 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.918139 15240 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.926955 11369 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:14.928364 15240 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.870494 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.923151 15217 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.929570 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.930209 12067 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:14.819130 15234 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:16:14.930871 15234 consensus_queue.cc:1243] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:16:14.818739 15239 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.835165 15183 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:16:14.907886 14783 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:14.887904 14638 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8.
I20260709 13:16:14.841737 15250 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:14.932288 15253 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:14.932703 15250 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:14.911590 15190 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.933650 15250 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.870200 15221 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:16:14.901005 11443 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:14.935469 15190 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.935644 11443 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:14.936426 11443 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.887081 15214 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.938722 15214 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.902794 15226 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.899621 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.908856 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.894899 11363 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:14.888336 14894 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.900228 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.889684 15150 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:14.903208 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:14.893640 15223 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:16:14.902974 15258 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.906507 15093 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:14.848388 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:14.939381 12066 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
W20260709 13:16:14.964674 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 116373us
W20260709 13:16:14.965591 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 250741us
I20260709 13:16:14.818182 15170 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:14.964811 11437 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:14.818845 15247 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.968044 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.968623 12067 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:14.832422 15220 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.966246 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:14.978163 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:14.979326 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:14.979528 12064 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:14.969310 15258 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.910094 14952 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:14.983197 12061 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:14.984705 14952 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:14.972245 14855 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:14.981151 15290 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 13:16:14.985914 15278 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 13:16:14.985975 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 170763us
I20260709 13:16:14.987885 11370 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:16:14.967602 15170 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:14.988391 15170 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.967722 15247 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:14.984985 15258 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:14.978401 15150 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 13:16:14.998734 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.987063 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:14.906720 15230 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:14.967232 15221 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:14.985908 14855 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.000846 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.002718 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:14.997316 15183 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:14.939637 15226 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.000234 15221 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:15.026021 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.068733 11512 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:15.026095 11516 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:16:15.070070 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.070779 11516 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:15.057711 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.027083 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.071575 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.071949 11515 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:15.073124 11516 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:15.073207 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.073468 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:15.058220 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 66768.6us
I20260709 13:16:15.074301 11515 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:15.012555 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.077051 11368 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:15.077937 12067 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:15.013283 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:15.078516 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 65303.8us
I20260709 13:16:15.076669 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.079753 11514 raft_consensus.cc:2393] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:15.081040 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.078715 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.081892 12067 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:15.082680 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.011615 15290 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:15.086423 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.056030 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:15.010753 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.086978 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.087189 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.094902 11518 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:15.085058 11515 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:15.056321 11519 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:15.013759 15183 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:15.012652 15293 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:15.101058 11366 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:16:15.085654 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:15.101416 11366 pending_rounds.cc:85] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 7
W20260709 13:16:15.092612 15296 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:16:15.024570 15180 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:15.085175 15223 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:16:15.096307 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.097054 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.122069 11408 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:15.123080 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.123688 11516 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.098635 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.125430 11514 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:15.058445 15221 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:15.125850 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.121788 11519 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.124353 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:15.099572 12060 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:15.099714 11517 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:15.085702 15173 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:16:15.096982 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:15.089531 15220 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:15.101722 11366 pending_rounds.cc:107] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:16:15.083066 15258 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:16:15.100606 15183 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:15.107668 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.109284 15248 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.110266 15226 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:15.025681 14783 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:16:15.100863 15293 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:15.112234 15093 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:15.011075 15190 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:16:15.093641 15290 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:15.084061 15298 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:15.114504 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:15.115109 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.116369 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.117004 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:15.117614 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:15.117846 14894 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:15.127486 11518 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:15.120613 15180 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.120465 15168 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:15.119311 15170 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.128186 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.133788 11366 raft_consensus.cc:2889] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:15.129688 15294 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:15.129367 15244 raft_consensus.cc:993] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:16:15.146111 11333 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:15.183912 15244 raft_consensus.cc:1081] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:16:15.184623 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:15.184806 15244 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:15.185215 15244 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:15.186106 15240 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:15.185945 15244 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.131420 15223 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:15.187091 11443 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:15.147326 11516 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:15.125528 15250 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:16:15.132892 15287 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:15.188272 15181 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:15.155004 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.153273 11362 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:15.189656 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.190358 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:15.190764 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:15.149948 15293 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.191433 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.191759 11333 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:15.192525 15173 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:15.193055 15173 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:15.193867 15173 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.180258 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.195655 11441 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:15.162110 15247 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.165277 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:15.172029 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.197366 11368 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:15.197539 11437 raft_consensus.cc:2376] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10. Re-sending same reply.
I20260709 13:16:15.198064 11369 raft_consensus.cc:2410] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:16:15.158849 11408 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:15.169478 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:15.192389 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:15.139901 15092 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:16:15.181787 15170 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:15.186530 15240 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.154672 11515 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:15.188813 15181 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:15.155556 15298 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.150341 15226 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:15.170271 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:15.178951 11335 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:15.200242 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.165756 11519 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:15.172686 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:15.206483 11443 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:15.209873 12067 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:15.220487 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:15.220811 15239 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.221275 12067 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:16:15.222142 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.222826 12067 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:15.209916 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.212718 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.224185 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2161) took 1215 ms. Trace:
I20260709 13:16:15.207698 15220 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.224550 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.217417 15226 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:15.225450 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:15.168373 15180 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:15.218029 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:15.218509 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.219323 11512 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:15.219213 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.183565 15294 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:15.231818 11366 raft_consensus.cc:2368] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:15.171010 15168 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:15.218358 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45767.3us
I20260709 13:16:15.224545 11513 rpcz_store.cc:276] 0709 13:16:14.008903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.351791 (+342888us) service_pool.cc:224] Handling call
0709 13:16:14.352068 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:14.352428 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:16:14.352718 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:16:14.469453 (+116735us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:14.469880 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:16:14.470008 (+   128us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:14.470040 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.470107 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:14.527714 (+ 57607us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b1800420100. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:15.222292 (+694578us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b1800420100. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:15.224004 (+  1712us) raft_consensus.cc:1685] finished
0709 13:16:15.224023 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:15.224131 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14147,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":908,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":43458,"spinlock_wait_cycles":15022720,"thread_start_us":43754,"threads_started":1,"wal-append.queue_time_us":89291}
I20260709 13:16:15.230527 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.228056 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.207016 14783 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.212757 15240 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:15.227646 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 48806.2us
I20260709 13:16:15.235759 14846 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:15.236480 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:15.238404 11363 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:15.241473 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.245471 11366 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:15.242630 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:15.245298 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:15.215315 15298 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.219712 15250 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:16:15.229506 15244 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.230141 14983 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.249114 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.251372 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:15.252506 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:15.248557 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:15.237886 15168 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.215368 15238 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:16:15.253491 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.234562 15233 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.214250 15113 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.254185 15225 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:15.258183 15190 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:15.254088 15087 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:15.271972 15087 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:15.272341 11440 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:16:15.272755 11440 pending_rounds.cc:85] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 4
I20260709 13:16:15.272861 15087 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.275925 11512 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:15.276486 15222 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:16:15.277343 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.278288 15293 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.278612 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.248243 14846 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:15.249459 14635 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:16:15.282971 12069 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:15.255939 15250 consensus_queue.cc:1243] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:16:15.286845 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 660987us
I20260709 13:16:15.287047 15220 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:15.287516 15220 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:15.269946 15150 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:16:15.289394 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:15.256544 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.290380 12067 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 4.
I20260709 13:16:15.290510 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:15.259063 11369 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:15.245787 15235 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:15.232856 15245 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:16:15.265897 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:15.247136 15248 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:15.253139 14848 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:15.280876 11440 pending_rounds.cc:107] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 5.5
W20260709 13:16:15.282035 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.282475 14846 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.264019 15170 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:16:15.285107 15113 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:15.286041 15301 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:15.279836 15183 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.225935 15226 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:15.310863 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 45020.5us
I20260709 13:16:15.314628 15173 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:15.267915 15225 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.287449 15237 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:15.323958 12063 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 3. Current term is 5. Ops: []
I20260709 13:16:15.307549 15247 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:15.288431 15220 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.323974 12062 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:15.328445 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:15.280355 15181 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:15.362996 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.363757 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:15.360119 11440 raft_consensus.cc:2889] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:15.258394 15290 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.312127 15093 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:15.472270 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.335664 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:15.415694 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 95190.9us
W20260709 13:16:15.437330 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 763824us
I20260709 13:16:15.559864 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:15.560385 12033 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:15.363492 15301 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:15.561215 15315 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:15.577124 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 248779us
I20260709 13:16:15.577883 11334 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:15.578213 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.579633 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.580284 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.582831 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.583376 11516 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:15.589959 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.537367 12067 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:15.590224 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.591429 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:15.592998 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.329413 15148 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:15.482714 15306 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:16:15.594587 15293 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:15.596177 15150 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.597676 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.601613 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:15.623662 15180 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.637970 15298 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.317420 15294 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.640444 15294 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:15.643186 15183 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:15.653411 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1166) took 1083 ms. Trace:
I20260709 13:16:15.653885 12066 rpcz_store.cc:276] 0709 13:16:14.570125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.583777 (+ 13652us) service_pool.cc:224] Handling call
0709 13:16:15.653194 (+1069417us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540077fbd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:15.653314 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800744320}
I20260709 13:16:15.655158 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:15.656111 12066 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:15.604538 15312 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:16:15.601651 11443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "857937fa61dc400a81345944f0a14511"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51484
I20260709 13:16:15.540338 15225 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:15.610901 15087 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.615566 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:15.607187 15247 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:15.619838 15248 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:15.620280 15170 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:15.363600 15221 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:15.562451 15301 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:15.613744 11516 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:15.691262 12066 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:15.608183 11483 leader_election.cc:400] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:15.621304 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:15.686527 11443 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:15.868767 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:15.693030 15226 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:15.629803 15180 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:16.211921 11443 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:15.732460 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:15.754739 11511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.006s	sys 0.015s
I20260709 13:16:15.870391 11483 leader_election.cc:403] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:16:16.212913 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 604826us
I20260709 13:16:16.213815 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:16.214843 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:15.706367 15290 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:16.219831 15148 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:16.219348 11443 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:16.222115 11443 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:15.877851 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 164529us
W20260709 13:16:16.224825 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.605859 15093 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:15.880429 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:16.225675 12062 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:16.226045 15093 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:16.227799 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:16.228456 11519 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:15.711771 15248 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:15.868418 15172 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.024s	sys 0.025s
W20260709 13:16:16.231472 15172 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.024s	sys 0.025s
I20260709 13:16:15.938504 11510 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:16:15.939788 15240 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.005s
I20260709 13:16:16.233145 15240 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:16.234308 11364 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:15.877345 14894 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:16.212487 11511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.006s	sys 0.021s
I20260709 13:16:16.235914 11516 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:16.236608 11514 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:15.869916 15113 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.237258 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:15.948591 15272 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
I20260709 13:16:15.951305 15315 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:16.238595 15293 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:15.869041 15230 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:15.959089 15298 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:15.796366 15170 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:16.239486 15230 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:16.241402 15170 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:15.972375 15168 consensus_meta.cc:287] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.000s
I20260709 13:16:16.241757 15230 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:16.242271 15168 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:16.236625 12066 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:16.244639 15168 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:15.976308 15223 consensus_meta.cc:287] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.000s
I20260709 13:16:16.245378 15223 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:16:15.978294 15244 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.724s	user 0.000s	sys 0.009s
W20260709 13:16:15.993484 15289 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.924s	user 0.001s	sys 0.002s
I20260709 13:16:16.247574 15325 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.009455 15301 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:16.237679 15113 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:16.247838 11519 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:16.250188 15113 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:16.019279 15234 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:16.250995 15234 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:16.253248 15234 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:16.103294 15328 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:16:16.140764 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 970) took 1597 ms. Trace:
W20260709 13:16:16.167683 15291 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.187s	user 0.006s	sys 0.000s
I20260709 13:16:16.253966 12065 rpcz_store.cc:276] 0709 13:16:14.543717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.545847 (+  2130us) service_pool.cc:224] Handling call
0709 13:16:16.127573 (+1581726us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54001fe854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:16.140685 (+ 13112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998558592}
I20260709 13:16:16.179728 15221 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:16.212267 11512 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:16.979031 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1689) took 1270 ms. Trace:
I20260709 13:16:16.979375 11512 rpcz_store.cc:276] 0709 13:16:15.708371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.708500 (+   129us) service_pool.cc:224] Handling call
0709 13:16:16.978947 (+1270447us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:16.197965 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2464) took 1084 ms. Trace:
I20260709 13:16:16.980588 11441 rpcz_store.cc:276] 0709 13:16:15.113863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.116626 (+  2763us) service_pool.cc:224] Handling call
0709 13:16:15.195492 (+ 78866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:15.196093 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.197753 (+1001660us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b54002ea750. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:16.197903 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180658560}
I20260709 13:16:16.194379 11363 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:16.257629 15325 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:16.208474 12060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 722) took 1421 ms (client timeout 1565 ms). Trace:
I20260709 13:16:16.248495 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 979) took 1685 ms. Trace:
W20260709 13:16:17.139133 12064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 977) took 2585 ms (client timeout 1902 ms). Trace:
I20260709 13:16:17.138280 12068 rpcz_store.cc:276] 0709 13:16:14.562963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.563083 (+   120us) service_pool.cc:224] Handling call
0709 13:16:14.567116 (+  4033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:14.619989 (+ 52873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:14.620022 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:14.652347 (+ 32325us) log.cc:844] Serialized 26 byte log entry
0709 13:16:14.965608 (+313261us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:14.965643 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.876799 (+911156us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54007783d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:15.876839 (+    40us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:15.992803 (+115964us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:16.248289 (+255486us) raft_consensus.cc:1685] finished
0709 13:16:16.248309 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.248433 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4130,"raft.queue_time_us":21617,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":9076,"spinlock_wait_cycles":82633984,"thread_start_us":248976,"threads_started":1,"wal-append.queue_time_us":322349}
I20260709 13:16:17.140270 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:17.140805 12068 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:17.141692 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.142205 12068 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:16.201061 12069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 984) took 1610 ms (client timeout 1586 ms). Trace:
W20260709 13:16:17.169298 12069 rpcz_store.cc:269] 0709 13:16:14.591053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.627284 (+ 36231us) service_pool.cc:224] Handling call
0709 13:16:16.200846 (+1573562us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54004b7154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:16.200989 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1395817344}
I20260709 13:16:17.170518 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:16.235819 14894 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.174933 14894 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:16.209019 15215 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.178545 15215 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:16.014252 15225 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:16.206254 15232 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:17.188474 15232 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:17.189127 15225 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:16.231231 15248 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:16.235286 15240 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:16.341778 15148 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
W20260709 13:16:16.355891 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 108650us
W20260709 13:16:17.204479 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.623683 11367 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:16.360237 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1776) took 1285 ms. Trace:
I20260709 13:16:17.207752 11370 rpcz_store.cc:276] 0709 13:16:15.075176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.075700 (+   524us) service_pool.cc:224] Handling call
0709 13:16:15.088692 (+ 12992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:15.089005 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:15.089037 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:16.350519 (+1261482us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b2c000c7fc8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:16.350950 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:16:16.359772 (+  8822us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:16.359811 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.359890 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:16.359936 (+    46us) raft_consensus.cc:1685] finished
0709 13:16:16.359949 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.360163 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":371,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":9975168,"thread_start_us":8430,"threads_started":1,"wal-append.queue_time_us":1078}
I20260709 13:16:17.209689 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:17.211552 15092 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:17.212116 15092 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:16.627601 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:16.854557 11438 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:16.869167 14848 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:16.873325 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 44999.8us
I20260709 13:16:17.214015 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 900) took 1639 ms. Trace:
I20260709 13:16:17.214396 11370 rpcz_store.cc:276] 0709 13:16:15.572420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.210699 (+1638279us) service_pool.cc:224] Handling call
0709 13:16:17.210972 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.211131 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.211155 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.211192 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:17.211221 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.211268 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.211371 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.215741 11519 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:16.887959 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2306) took 1715 ms. Trace:
I20260709 13:16:17.217234 11445 rpcz_store.cc:276] 0709 13:16:15.172840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.213316 (+ 40476us) service_pool.cc:224] Handling call
0709 13:16:15.213602 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:15.227157 (+ 13555us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:15.230245 (+  3088us) log.cc:844] Serialized 14 byte log entry
0709 13:16:15.230430 (+   185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:15.230476 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:15.230520 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.215674 (+985154us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b180033dd20. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:16.887678 (+672004us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b5400438fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:16.887735 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.887890 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3416,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1472,"raft.queue_time_us":2364,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6667,"spinlock_wait_cycles":40029312}
I20260709 13:16:16.902108 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1779) took 1825 ms. Trace:
I20260709 13:16:17.218998 11368 rpcz_store.cc:276] 0709 13:16:15.076403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.077877 (+  1474us) service_pool.cc:224] Handling call
0709 13:16:15.078126 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:15.208696 (+130570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.901837 (+1693141us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400165154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:16.902047 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273853184}
I20260709 13:16:17.219908 11367 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:17.220521 11367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 902) took 1634 ms (client timeout 1875 ms). Trace:
W20260709 13:16:16.911607 15301 consensus_meta.cc:287] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.663s	user 0.018s	sys 0.000s
I20260709 13:16:16.914852 15254 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:16.919374 11364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2318) took 1778 ms (client timeout 1918 ms). Trace:
I20260709 13:16:16.920938 11366 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:16:16.925216 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.936532 15287 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:16.942548 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2195) took 1694 ms. Trace:
I20260709 13:16:16.957018 11362 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:16:16.960040 15237 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.982144 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1782) took 1780 ms. Trace:
I20260709 13:16:17.225000 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:17.003989 15156 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.007241 15292 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.996s	user 0.004s	sys 0.010s
I20260709 13:16:17.008229 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2198) took 1404 ms. Trace:
W20260709 13:16:17.009016 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.432s	user 0.033s	sys 0.069s
W20260709 13:16:17.014865 12062 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.789s	user 0.011s	sys 0.002s
I20260709 13:16:17.021601 15314 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.026973 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:17.029646 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 743) took 2911 ms. Trace:
I20260709 13:16:17.030241 12033 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:17.033427 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:16:17.041924 12061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 971) took 2498 ms (client timeout 2260 ms). Trace:
I20260709 13:16:17.043548 15234 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:17.044010 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:17.044806 12066 consensus_meta.cc:287] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.800s	user 0.014s	sys 0.001s
I20260709 13:16:17.050680 15239 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:17.056267 14638 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:17.058334 15087 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.067795 15246 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:17.068344 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:17.069649 15245 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.005s	sys 0.012s
I20260709 13:16:17.081561 15231 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.088891 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.095541 15298 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:17.098866 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.102660 15220 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:17.109325 15308 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.417s	user 0.009s	sys 0.005s
I20260709 13:16:17.113603 15319 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:17.115063 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:17.278514 15222 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.278857 11369 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:17.120443 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:17.279163 15301 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:17.279857 11361 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:17.278367 15168 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:17.280520 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 903) took 1694 ms. Trace:
I20260709 13:16:17.279346 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 904) took 1692 ms. Trace:
I20260709 13:16:17.279786 15301 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.281131 11369 rpcz_store.cc:276] 0709 13:16:15.587041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.114823 (+1527782us) service_pool.cc:224] Handling call
0709 13:16:17.279286 (+164463us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.282735 15301 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.282574 15232 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.121475 15293 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.279006 15222 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.283501 15231 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.283692 15293 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.283962 11361 rpcz_store.cc:276] 0709 13:16:15.586484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.120231 (+1533747us) service_pool.cc:224] Handling call
0709 13:16:17.280462 (+160231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.285061 15244 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:17.285508 15231 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.130455 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:17.131819 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:17.137435 12060 rpcz_store.cc:269] 0709 13:16:14.787392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.818587 (+ 31195us) service_pool.cc:224] Handling call
0709 13:16:16.208287 (+1389700us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b540046d554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:16.208406 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403861376}
W20260709 13:16:17.286404 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 154637us
W20260709 13:16:17.139550 12064 rpcz_store.cc:269] 0709 13:16:14.553295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.553403 (+   108us) service_pool.cc:224] Handling call
0709 13:16:17.138912 (+2585509us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400522bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:17.139064 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948603264}
I20260709 13:16:17.285537 15244 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:17.279073 11263 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 5 to 8, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 8 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.287448 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.288152 12064 raft_consensus.cc:2368] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:17.153301 12068 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:17.288269 11513 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:17.277899 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.701s	user 0.035s	sys 0.071s
W20260709 13:16:17.160841 15290 consensus_meta.cc:287] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.945s	user 0.007s	sys 0.010s
W20260709 13:16:17.161926 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.290570 11444 rpcz_store.cc:276] 0709 13:16:14.118269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.343766 (+225497us) service_pool.cc:224] Handling call
0709 13:16:14.344098 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:14.344499 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:14.344527 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:14.345052 (+   525us) log.cc:844] Serialized 52 byte log entry
0709 13:16:14.504231 (+159179us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:14.541792 (+ 37561us) log.cc:844] Serialized 14 byte log entry
0709 13:16:14.562581 (+ 20789us) log.cc:844] Serialized 14 byte log entry
0709 13:16:14.571484 (+  8903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.571558 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:15.189260 (+617702us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b18003ba7c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:15.692333 (+503073us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b18003ba7c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:17.029424 (+1337091us) raft_consensus.cc:1685] finished
0709 13:16:17.029444 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.029590 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16131,"prepare.queue_time_us":689011,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":59,"raft.queue_time_us":9018,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":15740,"spinlock_wait_cycles":72111104,"thread_start_us":88885,"threads_started":1,"wal-append.queue_time_us":98291,"wal-append.run_cpu_time_us":9169,"wal-append.run_wall_time_us":1551863}
I20260709 13:16:17.163640 15170 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.165222 15093 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.011s	sys 0.005s
I20260709 13:16:17.180502 12067 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:16.255960 15221 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:17.186592 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.333942 12069 raft_consensus.cc:2368] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:16:17.334856 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:17.335446 12069 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:17.336199 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:17.337116 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.337702 12069 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:17.338861 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 987) took 1891 ms. Trace:
I20260709 13:16:17.187574 15215 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.213228 11510 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:17.221315 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:17.222445 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 752) took 1931 ms. Trace:
I20260709 13:16:17.223160 15092 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.341256 11438 rpcz_store.cc:276] 0709 13:16:15.290882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.605356 (+314474us) service_pool.cc:224] Handling call
0709 13:16:15.605608 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.213905 (+1608297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.222192 (+  8287us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003147d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:17.222382 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2871773056}
I20260709 13:16:17.220683 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 901) took 1644 ms. Trace:
W20260709 13:16:17.224968 11364 rpcz_store.cc:269] 0709 13:16:15.140993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.198476 (+ 57483us) service_pool.cc:224] Handling call
0709 13:16:16.919121 (+1720645us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540035b954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:16.919285 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323912960}
I20260709 13:16:16.983309 15325 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:17.270141 11362 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2316) took 2167 ms (client timeout 1928 ms). Trace:
I20260709 13:16:17.350345 11263 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 3 to 4, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.286686 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:17.351045 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf820 after lost signal to thread 10260
I20260709 13:16:17.352120 12060 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
W20260709 13:16:17.352684 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf740 after lost signal to thread 11302
I20260709 13:16:17.289103 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.277276 12062 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
W20260709 13:16:17.353564 12062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 994) took 1880 ms (client timeout 1768 ms). Trace:
W20260709 13:16:17.353837 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08000aee40 after lost signal to thread 11394
W20260709 13:16:17.354249 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08000f2060 after lost signal to thread 11395
W20260709 13:16:17.353897 12062 rpcz_store.cc:269] 0709 13:16:15.472667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.713570 (+240903us) service_pool.cc:224] Handling call
0709 13:16:17.353500 (+1639930us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:17.285794 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 60867.2us
I20260709 13:16:17.292572 12066 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:17.277566 11518 rpcz_store.cc:276] 0709 13:16:15.603802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.603937 (+   135us) service_pool.cc:224] Handling call
0709 13:16:15.604301 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:15.604668 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:15.604695 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:15.605136 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:16:16.013494 (+408358us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:16.013530 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.013594 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:16.254864 (+241270us) raft_consensus.cc:1685] finished
0709 13:16:16.254886 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.255028 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":978,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6219,"thread_start_us":408018,"threads_started":1,"wal-append.queue_time_us":416860,"wal-append.run_cpu_time_us":2551,"wal-append.run_wall_time_us":1236279}
W20260709 13:16:17.351302 11362 rpcz_store.cc:269] 0709 13:16:15.119807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.191210 (+ 71403us) service_pool.cc:224] Handling call
0709 13:16:17.269908 (+2078698us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54003228d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:17.270053 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4058963200}
W20260709 13:16:17.277134 12061 rpcz_store.cc:269] 0709 13:16:14.543992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.544948 (+   956us) service_pool.cc:224] Handling call
0709 13:16:17.041712 (+2496764us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400509954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:17.041856 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005783296}
I20260709 13:16:17.264343 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 905) took 1676 ms. Trace:
I20260709 13:16:17.294366 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 986) took 1847 ms. Trace:
W20260709 13:16:17.277992 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 247834us
I20260709 13:16:17.259225 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1781) took 2058 ms. Trace:
W20260709 13:16:17.296345 11334 connection.cc:256] RPC call timeout handler was delayed by -1.3411767999998574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
I20260709 13:16:17.228545 11363 rpcz_store.cc:276] 0709 13:16:15.201184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.262564 (+ 61380us) service_pool.cc:224] Handling call
0709 13:16:15.268881 (+  6317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.194224 (+925343us) spinlock_profiling.cc:243] Waited 97.8 ms on lock 0x7b5400236020. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc975dc0 00007f35bca001c0 00007f35bca01231 00007f35bc9fdd8f 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c
0709 13:16:16.981890 (+787666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.982085 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224989056}
I20260709 13:16:17.342702 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:16:17.296917 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 228636us
I20260709 13:16:17.357928 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:17.299012 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2474) took 2060 ms. Trace:
I20260709 13:16:17.298758 15087 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.298130 15314 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.284133 15298 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.300093 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:17.365233 11436 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:17.366300 15087 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:17.366492 15298 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.366987 12060 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:17.367911 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 988) took 1920 ms. Trace:
I20260709 13:16:17.368194 12060 rpcz_store.cc:276] 0709 13:16:15.447266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.352816 (+1905550us) service_pool.cc:224] Handling call
0709 13:16:17.353127 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.367341 (+ 14214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.367836 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.368722 12034 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:17.298511 15246 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.369271 12060 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:17.369982 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:17.371363 15246 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.278641 15319 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.302191 15248 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.371579 14952 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:17.372749 14952 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.373412 14952 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:17.298020 15156 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.285821 15293 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.373781 15248 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.374745 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:17.375213 15156 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.376214 15156 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:17.286001 14894 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:17.377656 14894 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.378086 14894 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:17.298614 14783 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:17.298677 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.224051 14848 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.381596 12064 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:17.381839 11362 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:17.382125 14848 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:17.303486 15093 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.012s	sys 0.005s
I20260709 13:16:17.382681 12064 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:17.383687 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.384112 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:17.384275 11438 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:17.384697 11364 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:17.385195 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.385636 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:17.305086 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.386269 11361 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:17.303128 14635 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.302139 15290 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:16:17.387262 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:17.306478 15170 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.300307 15182 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:17.303395 15088 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.388082 15314 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.280565 15168 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.389560 15170 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.390015 15168 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.308418 15232 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.307611 15213 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:17.306221 15234 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.306984 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.310348 11513 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:17.392498 15234 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.227998 15287 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:17.220876 11367 rpcz_store.cc:269] 0709 13:16:15.585906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.601311 (+ 15405us) service_pool.cc:224] Handling call
0709 13:16:17.220463 (+1619152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.308588 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:17.312880 15222 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.287748 15244 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.393409 15287 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.394421 15287 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:17.234280 11514 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:17.252463 15237 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.224625 15240 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:16:17.240832 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:17.227648 11515 rpcz_store.cc:276] 0709 13:16:15.248195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.248617 (+   422us) service_pool.cc:224] Handling call
0709 13:16:15.248885 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:16.007824 (+758939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:16.007857 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:16.008376 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:16:16.189599 (+181223us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:16.193485 (+  3886us) log.cc:844] Serialized 14 byte log entry
0709 13:16:16.205716 (+ 12231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.942102 (+736386us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b540031e554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:16.942153 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:16.942200 (+    47us) raft_consensus.cc:1685] finished
0709 13:16:16.942289 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.942462 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":240,"prepare.queue_time_us":1546,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":2244,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":2518,"spinlock_wait_cycles":1744953984,"thread_start_us":173557,"threads_started":1,"wal-append.queue_time_us":181421,"wal-append.run_cpu_time_us":2490,"wal-append.run_wall_time_us":1010161}
I20260709 13:16:17.319761 11443 pending_rounds.cc:85] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 3
I20260709 13:16:17.397642 11443 pending_rounds.cc:107] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 6.4
W20260709 13:16:17.321779 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 277832us
I20260709 13:16:17.398272 15293 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.398337 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:17.397816 15237 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.399883 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:17.400091 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:17.400632 11514 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:17.401011 11483 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:17.228420 15148 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:17.323939 11444 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:17.405262 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:17.227458 15225 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:16:17.328331 15239 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.341141 11512 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:17.339169 12069 rpcz_store.cc:276] 0709 13:16:15.447031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.338170 (+1891139us) service_pool.cc:224] Handling call
0709 13:16:17.338415 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.338580 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.338603 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.338641 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:17.338668 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.338716 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.338811 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.341400 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1175) took 2709 ms. Trace:
I20260709 13:16:17.423624 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:17.424321 11367 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:17.348905 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2200) took 1107 ms. Trace:
I20260709 13:16:17.124161 15233 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:17.294836 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.388020 15290 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:17.356782 11370 rpcz_store.cc:276] 0709 13:16:15.587593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.229153 (+1641560us) service_pool.cc:224] Handling call
0709 13:16:17.263780 (+ 34627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.263996 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.264021 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.264068 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:17.264102 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.264152 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.264270 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.357640 12063 rpcz_store.cc:276] 0709 13:16:15.446730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.355323 (+908593us) service_pool.cc:224] Handling call
0709 13:16:16.355670 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.173943 (+818273us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:17.182585 (+  8642us) log.cc:844] Serialized 14 byte log entry
0709 13:16:17.228687 (+ 46102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.228741 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:17.228785 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.275602 (+ 46817us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b180041fbc0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419
0709 13:16:17.294131 (+ 18529us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54005afad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:17.294198 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.294304 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6741,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":3948,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":18097,"spinlock_wait_cycles":1903828096,"thread_start_us":49982,"threads_started":2,"wal-append.queue_time_us":119323}
I20260709 13:16:17.350085 11368 rpcz_store.cc:276] 0709 13:16:15.576111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.219888 (+1643777us) service_pool.cc:224] Handling call
0709 13:16:17.220173 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.220351 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.220376 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.220444 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:17.220476 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.220530 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.220630 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.356455 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:17.359120 11334 connection.cc:256] RPC call timeout handler was delayed by -1.2092474469998251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
I20260709 13:16:17.362737 12065 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:17.363133 12062 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:17.357026 11366 rpcz_store.cc:276] 0709 13:16:15.201033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.246089 (+ 45056us) service_pool.cc:224] Handling call
0709 13:16:16.920682 (+1674593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.225278 (+304596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.258963 (+ 33685us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b540035b950. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:17.259151 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3850513664}
I20260709 13:16:17.228459 15254 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.358614 11442 rpcz_store.cc:276] 0709 13:16:15.238570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.239184 (+   614us) service_pool.cc:224] Handling call
0709 13:16:15.239517 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:15.239825 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:15.239851 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:15.240275 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:16:15.548142 (+307867us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:15.548185 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.548260 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:17.108789 (+1560529us) spinlock_profiling.cc:243] Waited 3.2 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:17.289522 (+180733us) raft_consensus.cc:1685] finished
0709 13:16:17.289546 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.298937 (+  9391us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7602,"raft.queue_time_us":1577,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":1015,"spinlock_wait_cycles":7368960,"thread_start_us":284430,"threads_started":1,"wal-append.queue_time_us":441148}
I20260709 13:16:17.295945 14855 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.385733 11438 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
W20260709 13:16:17.474648 11409 connection.cc:256] RPC call timeout handler was delayed by -1.3190794229999483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 13:16:17.386298 11439 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:16:17.475088 11409 connection.cc:256] RPC call timeout handler was delayed by -1.2092794229999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260709 13:16:17.396157 11263 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:17.311322 15245 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.006s	sys 0.013s
I20260709 13:16:17.290777 14638 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:17.323768 11437 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:17.401873 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.481261 11513 raft_consensus.cc:2368] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10.
I20260709 13:16:17.482097 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.482442 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 993) took 1993 ms. Trace:
I20260709 13:16:17.482769 11513 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
W20260709 13:16:17.475394 11409 connection.cc:256] RPC call timeout handler was delayed by -1.2020794229999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
I20260709 13:16:17.399158 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.399283 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 995) took 1908 ms. Trace:
W20260709 13:16:17.483439 11409 connection.cc:256] RPC call timeout handler was delayed by -1.0805794230000174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
I20260709 13:16:17.392868 11365 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:17.408700 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:17.483909 11519 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:17.373742 15319 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.410138 11443 raft_consensus.cc:2889] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:17.412581 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:17.418735 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.488430 11516 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:17.421142 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.489306 11517 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:17.422211 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:17.483004 12062 rpcz_store.cc:276] 0709 13:16:15.472420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.354829 (+1882409us) service_pool.cc:224] Handling call
0709 13:16:17.355076 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.465639 (+110563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.465878 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.490190 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.423851 12067 rpcz_store.cc:276] 0709 13:16:14.631846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.071216 (+1439370us) service_pool.cc:224] Handling call
0709 13:16:17.180279 (+1109063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.303813 (+123534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.341231 (+ 37418us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400509950. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:17.341348 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2549550848}
I20260709 13:16:17.424831 11436 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:17.424871 11510 rpcz_store.cc:276] 0709 13:16:16.233341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.627157 (+393816us) service_pool.cc:224] Handling call
0709 13:16:17.340876 (+713719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.422335 11514 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
W20260709 13:16:17.355295 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08000ebe20 after lost signal to thread 11469
I20260709 13:16:17.428771 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:17.387038 14635 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:17.464745 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 59542us
I20260709 13:16:17.434582 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.495849 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.431252 15345 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.430339 15148 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.421643 15087 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.473253 15290 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:17.430531 15239 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:17.480381 15093 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:17.409813 12064 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:17.483708 12061 rpcz_store.cc:276] 0709 13:16:15.490454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.356826 (+1866372us) service_pool.cc:224] Handling call
0709 13:16:17.385722 (+ 28896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.397240 (+ 11518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.397270 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.397310 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:17.397339 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.399059 (+  1720us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5400751054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:17.399105 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.399231 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26036096}
I20260709 13:16:17.419762 15234 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:17.487815 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 116452us
I20260709 13:16:17.492152 11515 raft_consensus.cc:2393] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:17.491219 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.509362 11440 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:17.355803 12066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 992) took 1883 ms (client timeout 1855 ms). Trace:
W20260709 13:16:17.510000 12066 rpcz_store.cc:269] 0709 13:16:15.472161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.732185 (+260024us) service_pool.cc:224] Handling call
0709 13:16:17.355738 (+1623553us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:17.494259 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 66338.9us
I20260709 13:16:17.398937 15293 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.496618 11513 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:16:17.495607 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08002e78c0 after lost signal to thread 11543
I20260709 13:16:17.498687 15213 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:17.497812 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 725) took 2709 ms. Trace:
I20260709 13:16:17.498185 11511 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:17.388445 15088 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.479290 15182 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:17.480206 15232 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.502178 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.493191 15113 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.488224 15233 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:16:17.479770 11262 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39460 (request call id 365) took 17081 ms (client timeout 15000 ms). Trace:
I20260709 13:16:17.487195 15092 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.433969 14952 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:16:17.487046 15325 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.561210 15325 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.562480 15293 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:17.563012 15293 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:17.484462 15168 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:17.563459 15293 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.511858 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:17.563951 12064 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:17.497582 14783 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:17.563566 15092 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.497403 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:17.528576 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 172265us
I20260709 13:16:17.567363 15092 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:17.530596 11466 debug-util.cc:398] Leaking SignalData structure 0x7b08003e14e0 after lost signal to thread 12093
W20260709 13:16:17.568068 11466 debug-util.cc:398] Leaking SignalData structure 0x7b080036cc60 after lost signal to thread 12094
I20260709 13:16:17.529620 15087 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.567171 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 69860.3us
I20260709 13:16:17.570374 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:17.500313 15345 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.541980 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.542308 11442 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:17.540884 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.536070 12065 rpcz_store.cc:276] 0709 13:16:14.788214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.254949 (+1466735us) service_pool.cc:224] Handling call
0709 13:16:17.355480 (+1100531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.464895 (+109415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.497651 (+ 32756us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400658e50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:17.497762 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2530171776}
I20260709 13:16:17.544456 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.495568 14775 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:16:17.484644 14855 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.577977 11439 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:17.553192 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.555343 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:17.552248 11262 rpcz_store.cc:269] 0709 13:16:00.384019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.384225 (+   206us) service_pool.cc:224] Handling call
0709 13:16:17.464968 (+17080743us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:15.111701 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:15.112245 (+   544us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:15.112282 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:16:15.112300 (+    18us) tablet.cc:662] Decoding operations
0709 13:16:15.113532 (+  1232us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:15.113584 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:15.113609 (+    25us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:16:15.114664 (+  1055us) tablet.cc:701] Row locks acquired
0709 13:16:15.114684 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:16:15.114817 (+   133us) write_op.cc:270] Start()
0709 13:16:15.117072 (+  2255us) write_op.cc:276] Timestamp: P: 1783602975114769 usec, L: 0
0709 13:16:15.117096 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:16:15.211335 (+ 94239us) log.cc:844] Serialized 5881 byte log entry
0709 13:16:16.029626 (+818291us) op_driver.cc:464] REPLICATION: finished
0709 13:16:16.191985 (+162359us) write_op.cc:301] APPLY: starting
0709 13:16:16.192086 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:16.192392 (+   306us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:16.192528 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:16.194691 (+  2163us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:16.194947 (+   256us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:16:16.194994 (+    47us) write_op.cc:312] APPLY: finished
0709 13:16:16.195310 (+   316us) log.cc:844] Serialized 107 byte log entry
0709 13:16:16.195498 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:16.195650 (+   152us) write_op.cc:454] Released schema lock
0709 13:16:16.195703 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:16.196038 (+   335us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:16.196065 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":325189,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":95460,"apply.run_cpu_time_us":3687,"apply.run_wall_time_us":4056,"num_ops":15,"prepare.queue_time_us":344927,"prepare.run_cpu_time_us":26255,"prepare.run_wall_time_us":247353,"raft.queue_time_us":429434,"raft.run_cpu_time_us":11003,"raft.run_wall_time_us":162715,"replication_time_us":912350,"thread_start_us":838608,"threads_started":4,"wal-append.queue_time_us":196764,"wal-append.run_cpu_time_us":47709,"wal-append.run_wall_time_us":1784179}]]}
I20260709 13:16:17.582458 12063 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:17.577425 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:17.556960 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:17.501317 15223 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:16:17.557466 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.548826 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.583739 11441 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:17.559675 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:17.560566 15346 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:17.510483 11263 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.559607 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:17.561007 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:17.581967 11438 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:17.577386 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:17.543928 15213 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:17.583195 12067 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:17.549031 15234 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:17.585176 11436 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:17.597577 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.599804 12062 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:17.510141 15093 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:17.622135 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 110376us
I20260709 13:16:17.531623 15229 consensus_queue.cc:1059] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:16:17.623687 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:17.563279 15325 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:17.624873 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 49305.6us
I20260709 13:16:17.628340 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:17.628937 12032 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:17.628280 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50973.4us
I20260709 13:16:17.630360 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:17.631420 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 222776us
I20260709 13:16:17.600322 15213 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:17.633487 14783 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:17.633788 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:17.639603 11444 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:16:17.568724 15182 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.621582 15234 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.579350 15168 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:17.637564 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 235581us
I20260709 13:16:17.598151 15222 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:17.638943 11263 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 5 to 8, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 8 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.634475 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.640007 11408 leader_election.cc:400] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10.
I20260709 13:16:17.552752 15232 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.578729 14848 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:16:17.546610 14638 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.640630 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:17.569289 15290 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.598819 15087 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:17.857987 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:17.858759 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.655506 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:17.857625 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 233989us
I20260709 13:16:17.655203 14894 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.621784 15233 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:17.652043 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 989) took 2204 ms. Trace:
I20260709 13:16:17.632586 15223 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:17.632406 15314 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:17.657560 15246 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.216s
I20260709 13:16:17.640836 15254 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.643718 15316 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:17.858681 15092 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:17.874408 15316 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:17.864779 15231 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:17.874943 15246 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:17.863655 11263 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 reported cstate change: leader changed from <none> to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.648229 15093 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.875286 15316 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.645185 15220 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:17.877681 11263 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 4 to 7, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.878434 15220 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:17.864602 14894 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.673015 15244 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.644536 15170 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:17.880036 15170 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:17.880868 15170 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.882134 15170 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:17.660161 15113 raft_consensus.cc:3055] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:17.887873 15113 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:17.888779 15113 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.641841 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:17.890937 14894 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:17.891328 14894 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.891662 14894 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:17.893062 15239 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.895118 15239 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:17.649923 15088 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:17.915310 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 49015us
I20260709 13:16:17.861163 11518 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:17.860801 11408 leader_election.cc:403] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:17.861526 11440 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
W20260709 13:16:17.868290 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 227715us
I20260709 13:16:17.916970 11437 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:17.865204 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 209828us
I20260709 13:16:17.918360 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:17.642064 15319 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:16:17.868645 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 300325us
I20260709 13:16:17.867211 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:17.658461 14783 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.643478 15148 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:17.638643 15305 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:17.890182 15087 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.895787 15290 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:17.896130 15287 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:17.913695 15314 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:17.931313 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.899677 11263 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:17.900692 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 258907us
W20260709 13:16:17.865123 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 386502us
I20260709 13:16:17.727490 12062 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:17.642908 15293 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:17.736477 11512 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:17.935914 11510 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
W20260709 13:16:17.865453 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 237175us
I20260709 13:16:17.866008 15345 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:17.890153 15113 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:17.869688 12068 rpcz_store.cc:276] 0709 13:16:15.447529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.353749 (+1906220us) service_pool.cc:224] Handling call
0709 13:16:17.354043 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.354447 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.354488 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:17.354963 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:16:17.392095 (+ 37132us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b18004be7a0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:17.494429 (+102334us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:17.494462 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.494520 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:17.651762 (+157242us) raft_consensus.cc:1685] finished
0709 13:16:17.651781 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.651987 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1231,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":7406,"spinlock_wait_cycles":52079872,"thread_start_us":102178,"threads_started":1,"wal-append.queue_time_us":187095}
I20260709 13:16:17.919130 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:17.920660 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 53521.5us
I20260709 13:16:17.922326 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:17.922869 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 286590us
I20260709 13:16:17.942777 11437 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:17.860575 15234 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:17.923606 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.945616 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.945868 11516 raft_consensus.cc:2393] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:17.946455 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:17.919289 15222 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:17.638716 15325 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.924220 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:17.925896 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.927471 15237 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:17.929241 14638 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:17.867838 14635 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.641198 15359 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:17.927959 11513 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:17.658073 15298 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.269s
I20260709 13:16:17.945178 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:17.926445 15231 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:16:17.923738 15245 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:17.926333 14775 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:17.949343 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:17.950326 11511 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:17.919785 12034 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:17.951654 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:17.951550 11263 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.953534 11361 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:17.657721 14855 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.879352 15220 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:17.954902 15237 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.927452 15305 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:17.957329 11370 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:16:17.958986 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.985648 11512 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:17.920181 15319 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.961714 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:17.943159 15314 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:17.948354 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:17.924729 14894 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.967549 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:17.968600 11369 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:17.941565 15287 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:17.947835 15222 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:17.970084 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 45873.9us
I20260709 13:16:17.970036 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:17.971729 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:17.954980 15187 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.957113 15148 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:17.983177 11483 leader_election.cc:400] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:17.982951 11263 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 8, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 8 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.964298 15362 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:17.999737 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.002185 11366 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:17.972499 15325 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:17.970520 15213 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.030212 15213 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:18.000934 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.980396 14848 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:16:17.982652 15358 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:18.002987 15319 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:18.001241 15239 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:18.031906 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.030311 11263 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:17.999675 15231 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:17.984578 15237 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:18.009282 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.060364 15358 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:18.065131 14783 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:18.065425 15358 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:18.066256 14894 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:17.998208 15220 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:18.023278 11483 leader_election.cc:403] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:16:18.017108 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:18.003515 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:18.036285 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 61049.7us
I20260709 13:16:18.037751 15168 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:17.972347 15233 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:18.038731 15356 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:16:18.027619 15187 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.020632 15248 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:18.021047 15287 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:18.037796 15232 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:18.033504 15362 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:17.970870 15359 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:18.026324 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 391663us
I20260709 13:16:18.058233 15357 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:18.060325 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:18.071244 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 54208.1us
I20260709 13:16:18.111814 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:18.112874 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:18.112950 11363 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:18.111559 11368 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
W20260709 13:16:18.071171 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 88111us
I20260709 13:16:18.114467 11483 leader_election.cc:400] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:16:18.115207 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:18.071687 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 137155us
I20260709 13:16:18.078004 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:18.088609 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 168900us
I20260709 13:16:18.061326 15231 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.110992 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:18.095288 14775 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:18.060087 15148 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:18.062892 15244 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:16:18.070734 15325 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.084089 15293 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:18.116647 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:18.109531 15170 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:18.122706 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.145090 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.145344 11445 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:16:18.124929 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:18.146009 11510 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 5: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:18.138913 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:18.113857 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:18.120687 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:18.107436 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 103993us
I20260709 13:16:18.129495 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.051749 15305 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:18.156560 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:18.157734 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:18.157819 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:18.158252 11516 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:18.125773 15231 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:18.126217 15240 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:18.137149 15362 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:18.117475 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 57234.7us
I20260709 13:16:18.138831 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.138536 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.151824 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.151173 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:18.105417 14848 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:16:18.215106 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 46444.5us
I20260709 13:16:18.154130 11367 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:16:18.147184 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:18.110817 15357 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:18.135066 15148 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.127981 15232 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:18.127810 15168 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:18.166414 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:18.166658 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49871.6us
I20260709 13:16:18.166742 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.178454 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.144606 15170 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:18.175238 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.166307 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.208200 11361 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:18.240679 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.241444 11361 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:16:18.208607 15384 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:18.209156 11363 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:18.165818 15248 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:18.209793 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:18.246086 11436 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:18.137877 15092 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.239499 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:18.152743 11483 connection.cc:256] RPC call timeout handler was delayed by -1.8682366729999558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
I20260709 13:16:18.210466 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:18.249396 11483 connection.cc:256] RPC call timeout handler was delayed by -1.8592366729999412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
I20260709 13:16:18.249599 12061 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:18.125990 14775 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:18.172438 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.210155 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.135125 15088 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:18.132983 15290 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:18.138060 15223 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.222222 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:18.227584 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.144647 14894 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:18.125948 15239 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:16:18.306365 11511 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:18.106923 15361 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
W20260709 13:16:18.314738 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 92627.4us
I20260709 13:16:18.232007 11364 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:18.232081 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:18.205065 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 143916us
I20260709 13:16:18.316499 12067 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:18.316665 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:18.317866 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:18.209756 15367 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.204082 15240 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:18.209524 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88910.1us
I20260709 13:16:18.234086 12063 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:18.233774 12062 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:16:18.324102 11436 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:18.238615 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:18.325769 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:18.242266 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.217975 15375 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:18.250613 11370 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:16:18.248375 15248 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.243247 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.249058 11369 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:18.154129 15325 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.218101 15368 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:18.153554 15222 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.234700 15305 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:18.261325 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.233245 15168 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:18.277552 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.242530 15170 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:18.233345 15373 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.245554 15384 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.278380 15246 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:16:18.323009 12060 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:18.254462 11516 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:18.255575 14775 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:18.304299 15223 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:18.357371 12066 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:18.257266 15092 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:18.366765 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.239454 15374 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.159s
I20260709 13:16:18.332787 15093 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:18.405869 11362 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:18.376943 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.407835 11369 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:16:18.408839 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.355284 15370 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:18.369843 11439 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:18.369935 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44232.1us
W20260709 13:16:18.355520 15371 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:16:18.373163 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:18.331028 15240 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:18.375743 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:18.333645 15254 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:18.425689 11363 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:18.373031 15384 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:18.420709 11436 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:16:18.430863 11436 pending_rounds.cc:85] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 5
I20260709 13:16:18.431137 11436 pending_rounds.cc:107] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:16:18.422335 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:18.376186 15298 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:18.427954 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54881.9us
I20260709 13:16:18.322392 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.365231 15168 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:18.433053 11365 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:18.440042 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.440833 11369 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:16:18.431541 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 153638us
I20260709 13:16:18.432168 11436 raft_consensus.cc:2889] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:18.433373 11442 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:18.437683 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:18.442588 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:18.438920 11439 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:18.449991 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:18.448405 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.486128 11516 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:18.487977 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:18.450913 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.490688 11442 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:16:18.431245 15381 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:16:18.330261 15357 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.489462 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.376372 15345 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:16:18.330799 15367 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:18.432672 15298 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:18.496232 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.464206 11257 maintenance_manager.cc:419] P ab9dad95b38845f197fb7c3c12a492c4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033374
W20260709 13:16:18.504602 11514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.002s	sys 0.014s
W20260709 13:16:18.500277 15369 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.378s	user 0.001s	sys 0.000s
I20260709 13:16:18.435751 15215 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:18.432389 15232 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:18.372725 15373 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.467207 11363 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:18.513146 15368 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:18.516665 11514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.003s	sys 0.015s
I20260709 13:16:18.507180 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.487516 15187 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:18.517123 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:18.503751 15113 consensus_meta.cc:287] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.000s
I20260709 13:16:18.509968 11366 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:18.587036 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:18.587667 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 139315us
I20260709 13:16:18.496318 15370 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:18.511476 14783 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.517051 15298 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:18.589526 15370 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.583091 15215 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:18.601142 15359 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:18.619557 15215 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.601013 15391 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:16:18.642169 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 308774us
I20260709 13:16:18.602969 15248 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:18.645159 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:18.613149 15358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.017s	sys 0.005s
I20260709 13:16:18.654814 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:18.656044 12062 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:18.657680 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.658404 12062 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:18.660255 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.662191 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.663798 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.666258 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:18.668655 15233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.017s	sys 0.026s
W20260709 13:16:18.669085 15233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.017s	sys 0.026s
I20260709 13:16:18.616072 15187 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:18.613572 15370 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:18.601707 15387 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:18.675084 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.681999 12066 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:18.682542 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:18.683053 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 535904us
I20260709 13:16:18.684671 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:18.691174 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:18.692035 12061 raft_consensus.cc:2376] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9. Re-sending same reply.
I20260709 13:16:18.694936 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.695190 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.695505 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.695731 11366 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:18.695973 11363 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:18.696127 11362 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:18.696168 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.700217 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.700655 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.700788 11362 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:18.700693 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.701678 11363 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
W20260709 13:16:18.702554 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 57472.6us
I20260709 13:16:18.704787 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:18.690984 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:18.705579 12063 raft_consensus.cc:2376] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8. Re-sending same reply.
W20260709 13:16:18.673370 15358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.017s	sys 0.005s
I20260709 13:16:18.706923 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:18.709012 15223 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:18.709630 14894 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.711992 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:18.603595 14783 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.718858 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:18.720844 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:18.728839 11334 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:18.640537 15391 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:18.730907 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 140847us
I20260709 13:16:18.734380 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:18.737021 11441 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.005s
W20260709 13:16:18.737516 11441 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.002s	sys 0.005s
I20260709 13:16:18.739827 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:18.681458 15387 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:18.706482 15187 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:18.749507 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:18.750068 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:18.753576 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:18.750806 11441 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:18.731545 14635 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:18.726184 15361 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:18.764008 15401 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:18.764559 15401 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:18.764951 15401 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:18.766927 15220 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.679s	user 0.008s	sys 0.000s
I20260709 13:16:18.756659 15359 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:18.751528 15092 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:18.771740 15223 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:18.748283 15244 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:18.774672 15170 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.738468 14894 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.777762 14894 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:18.777882 15314 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.745s	user 0.003s	sys 0.004s
I20260709 13:16:18.778407 15314 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.781338 15170 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:18.766927 15234 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.809s	user 0.000s	sys 0.006s
I20260709 13:16:18.782404 15234 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:18.636183 15232 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.783654 11519 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:18.784246 15367 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:18.784487 15234 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:18.785662 15393 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:16:18.786180 11438 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:18.743307 15248 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:18.782774 15231 consensus_meta.cc:287] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.000s
I20260709 13:16:18.755949 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:18.779287 15319 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.717s	user 0.003s	sys 0.005s
I20260709 13:16:18.752629 15387 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:18.805423 15355 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:16:18.806851 14775 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.807271 15187 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.712705 15368 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.808920 11442 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
W20260709 13:16:18.797819 15245 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.820s	user 0.007s	sys 0.003s
W20260709 13:16:18.798844 15287 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.000s
I20260709 13:16:18.799069 15234 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:16:18.817955 15168 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.818689 14783 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:18.771274 15092 consensus_queue.cc:1059] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:16:18.789158 15361 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.841729 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.844097 11442 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:18.824107 15391 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.796330 15382 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:18.827193 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:18.831154 15237 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.755s	user 0.004s	sys 0.005s
I20260709 13:16:18.831691 15234 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:18.800603 15170 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:18.841048 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:18.808043 15222 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:18.853596 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:18.815250 15220 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:18.868625 11440 raft_consensus.cc:2368] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:18.875553 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:18.884969 11439 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:18.827037 15240 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.835093 15245 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10.
I20260709 13:16:18.800834 15231 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.834255 15407 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:16:18.873157 15410 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:18.865021 14783 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:18.813081 15319 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.869449 14775 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:18.911738 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 207263us
I20260709 13:16:18.908644 15396 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:18.887264 15220 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:18.836205 15287 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.884631 15234 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:18.913496 15305 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
I20260709 13:16:18.891944 15382 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:18.910055 15170 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:18.911995 15092 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:18.872321 15237 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:18.912431 14783 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:18.915926 15222 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:19.078787 15361 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:18.919711 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:18.912223 15418 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:18.920960 15234 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:18.914189 15407 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:19.095088 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 175470us
I20260709 13:16:18.919481 15415 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:19.081852 15384 consensus_meta.cc:287] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 13:16:19.175220 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:19.196110 11439 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.138759 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:19.139189 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:19.201740 11441 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:19.127380 15298 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.001s
I20260709 13:16:19.236369 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:19.175748 15416 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 13:16:18.922504 15396 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.250979 11439 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:19.124539 15287 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.257385 15373 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.107924 15305 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:19.210103 15384 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:19.185391 15314 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.261693 15427 consensus_queue.cc:1059] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:16:19.125733 15319 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.263425 15391 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:19.122144 15382 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.175170 15187 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.129537 14783 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:19.179906 15170 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.096340 15370 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.180840 15387 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.295702 15387 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.123245 15237 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:19.297667 15384 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.298630 11445 raft_consensus.cc:2368] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
I20260709 13:16:19.091176 15092 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.093520 15222 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:19.101678 15418 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:19.270817 11484 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.000s	sys 0.013s
I20260709 13:16:19.177856 15168 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.178779 15240 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.124508 15231 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.122474 15388 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:19.248042 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:19.322198 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 74242.8us
I20260709 13:16:19.322875 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:19.324059 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:19.275028 12065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.007s	sys 0.008s
W20260709 13:16:19.275295 11483 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.011s	sys 0.005s
W20260709 13:16:19.326061 11483 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.011s	sys 0.005s
W20260709 13:16:19.326381 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 592062us
I20260709 13:16:19.293552 15187 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.246191 15405 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.327092 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:19.327250 15405 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.329417 15405 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.250927 15368 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:19.329442 15370 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.282238 11441 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:16:19.283064 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.035s	sys 0.053s
I20260709 13:16:19.270328 15088 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:19.163815 15220 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.267505 15298 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.223966 15415 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.305500 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1187) took 1139 ms. Trace:
I20260709 13:16:19.309301 15093 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:19.309587 15418 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.316874 12068 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:16:19.318359 15365 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:19.358153 15365 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:19.319240 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 769) took 1081 ms. Trace:
I20260709 13:16:19.359685 11437 rpcz_store.cc:276] 0709 13:16:18.237451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.237616 (+   165us) service_pool.cc:224] Handling call
0709 13:16:18.249024 (+ 11408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.598252 (+349228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.601924 (+  3672us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.614130 (+ 12206us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.704329 (+ 90199us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:18.704373 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.704445 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:19.319018 (+614573us) raft_consensus.cc:1685] finished
0709 13:16:19.319040 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.319175 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":357926,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":9985,"thread_start_us":367883,"threads_started":2,"wal-append.queue_time_us":607770}
W20260709 13:16:19.306784 11484 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.000s	sys 0.013s
W20260709 13:16:19.332782 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.035s	sys 0.053s
W20260709 13:16:19.361428 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 611130us
I20260709 13:16:19.364600 15358 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:19.365172 15358 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.366070 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.366390 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 871) took 1210 ms. Trace:
I20260709 13:16:19.367560 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:19.367736 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.368718 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.368830 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.369544 11517 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:19.350431 12060 rpcz_store.cc:276] 0709 13:16:18.166085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.209922 (+ 43837us) service_pool.cc:224] Handling call
0709 13:16:19.305289 (+1095367us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b54007783d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:19.305430 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166138112}
I20260709 13:16:19.370736 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.371359 12063 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:19.372301 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2491) took 1206 ms. Trace:
I20260709 13:16:19.372691 11438 rpcz_store.cc:276] 0709 13:16:18.165866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.166085 (+   219us) service_pool.cc:224] Handling call
0709 13:16:19.372140 (+1206055us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54002783d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:19.372241 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255717248}
I20260709 13:16:19.374054 11438 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:19.374763 15401 consensus_meta.cc:287] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.609s	user 0.004s	sys 0.003s
W20260709 13:16:19.325944 12065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.007s	sys 0.008s
I20260709 13:16:19.349318 15088 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:19.356040 15087 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:19.356386 15298 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.352931 15093 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:19.356684 15415 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:19.357468 15396 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.365423 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.369541 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:19.370100 12062 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:19.352070 12034 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:19.377007 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.416584 11437 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:19.354101 15392 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:19.325202 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:19.383826 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:19.379516 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.385954 11518 raft_consensus.cc:2368] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:16:19.366823 11511 rpcz_store.cc:276] 0709 13:16:18.156269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.157613 (+  1344us) service_pool.cc:224] Handling call
0709 13:16:19.366191 (+1208578us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000654d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:19.366326 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339848064}
I20260709 13:16:19.367105 15358 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.332108 15388 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:19.336200 15168 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:19.432161 11514 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:19.391266 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:19.353010 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:19.355270 15220 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.337287 15387 raft_consensus.cc:2764] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:16:19.397078 11436 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.419847 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:19.436265 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:19.375260 11438 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:19.352999 15239 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:19.437314 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:19.422353 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.376204 15432 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:19.385677 15391 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.315940 15234 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.431807 14775 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.431874 11512 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:19.458441 15430 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:19.433081 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.459492 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.458468 14775 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.460400 11512 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:19.433990 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 108861us
W20260709 13:16:19.435199 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 82273.2us
I20260709 13:16:19.461763 15092 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.395180 15411 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:19.462556 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:19.465425 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:19.466230 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:19.385788 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:19.438237 11510 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:19.467842 11512 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:19.438912 15384 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.439633 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:19.438941 15374 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:19.443668 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.469157 11513 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.443732 15093 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.443645 15392 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:19.445642 15388 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.446516 15406 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.447006 15168 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:19.448006 15325 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.448348 15431 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:19.437589 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:19.452129 11436 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
W20260709 13:16:19.455128 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 103145us
I20260709 13:16:19.463532 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:19.472671 11516 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:19.479223 11510 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:19.489905 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:19.491154 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:19.493010 11438 raft_consensus.cc:2368] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:19.494254 12034 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:19.496759 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.497856 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:19.500550 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:19.512079 12062 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:19.508667 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:19.515097 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:19.516018 11436 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:19.516824 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.521976 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:19.537257 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:19.538576 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:19.532019 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:19.540688 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:19.470781 15374 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.541972 11438 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 5: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:19.470708 15405 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.480538 15087 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.461809 14775 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.375160 15401 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.482543 15239 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:19.470801 15430 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:19.546058 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:19.545413 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:19.550208 11513 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:19.553046 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:19.544217 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:19.555558 11516 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:19.568022 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.325498 15407 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:19.545347 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:19.507506 15388 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.536170 15368 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:19.580819 12062 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:16:19.536339 15325 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:19.582392 12065 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:19.579543 12067 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:19.541591 15113 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:19.422043 15222 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.532316 15232 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:19.585654 11513 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:19.473254 15391 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.462598 15411 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.584599 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.586488 15092 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:19.586728 15404 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:19.472946 15432 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.550729 15182 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.542829 15305 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:19.553146 15374 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:19.545572 15427 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:19.598289 12069 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:19.581796 15187 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:19.484776 15406 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.503667 15431 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.586512 15400 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:19.605253 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:19.640785 12063 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:19.608695 11514 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:19.589991 15325 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:19.613480 12069 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:19.613619 15093 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.600091 15113 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:19.610633 15182 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.621896 15432 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.623528 15240 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:19.631948 11518 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:19.636480 15088 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:19.587715 15368 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:19.585987 15239 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.568125 15405 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.653347 11515 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.655349 11518 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:19.646512 15404 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:19.647730 15325 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:19.648157 15246 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:19.600301 15392 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:19.659547 15368 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:19.607151 15168 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.665400 15404 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:19.687602 15087 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:19.669644 15246 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:19.671906 15370 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.688228 15113 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.700661 15405 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.704303 15182 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:19.705614 15088 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:19.705670 15418 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.682812 15234 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.656064 15430 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.712779 15401 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.714347 15391 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.689662 15404 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:19.693811 15246 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:19.697952 15370 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.765158 15431 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.767477 15411 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.768685 15418 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:19.809708 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 384984us
I20260709 13:16:19.636752 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2331) took 1147 ms. Trace:
I20260709 13:16:19.812196 11445 rpcz_store.cc:276] 0709 13:16:18.488988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.489191 (+   203us) service_pool.cc:224] Handling call
0709 13:16:19.636518 (+1147327us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b54000843d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:19.636679 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1828518144}
I20260709 13:16:19.600370 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:19.818006 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 217602us
W20260709 13:16:19.818245 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:19.824337 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 438799us
I20260709 13:16:19.826714 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:19.832783 12060 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:19.837430 15092 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:19.838155 15092 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:19.842190 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:19.845115 15092 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.854698 15087 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:19.857271 15087 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:19.857972 15087 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:19.859611 15087 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:19.861508 11408 leader_election.cc:400] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:16:19.862111 11408 leader_election.cc:403] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:16:19.862712 11519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2212) took 1938 ms (client timeout 2183 ms). Trace:
W20260709 13:16:19.863258 11519 rpcz_store.cc:269] 0709 13:16:17.924351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.924468 (+   117us) service_pool.cc:224] Handling call
0709 13:16:19.862204 (+1937736us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54002c0954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:19.862465 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1953805696}
W20260709 13:16:19.871547 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.884847 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:19.889051 15240 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:19.891471 15240 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:19.894364 15240 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:19.898223 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2488) took 1771 ms. Trace:
I20260709 13:16:19.896492 15223 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:19.904410 15223 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:19.905023 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1015) took 1236 ms. Trace:
I20260709 13:16:19.905498 12068 rpcz_store.cc:276] 0709 13:16:18.668230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.669082 (+   852us) service_pool.cc:224] Handling call
0709 13:16:18.669446 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:19.353884 (+684438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.904412 (+550528us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54004a6fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:19.904820 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488418688}
I20260709 13:16:19.889429 14775 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:19.920795 14775 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:19.898810 11444 rpcz_store.cc:276] 0709 13:16:18.126915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.127080 (+   165us) service_pool.cc:224] Handling call
0709 13:16:18.154577 (+ 27497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.649825 (+495248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.649865 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.650249 (+   384us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.735777 (+ 85528us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:18.735817 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.897341 (+1161524us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54003151d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:19.897435 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:19.897524 (+    89us) raft_consensus.cc:1685] finished
0709 13:16:19.897541 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.897977 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":322648,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6661,"spinlock_wait_cycles":617027968,"thread_start_us":311784,"threads_started":2,"wal-append.queue_time_us":218118}
I20260709 13:16:19.895150 15240 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:19.900671 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2486) took 1778 ms (client timeout 2071 ms). Trace:
I20260709 13:16:19.978246 15392 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7.
I20260709 13:16:19.982785 15370 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:19.983071 15444 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:19.973142 11443 rpcz_store.cc:269] 0709 13:16:18.121963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.122184 (+   221us) service_pool.cc:224] Handling call
0709 13:16:19.900112 (+1777928us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002099d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:19.900431 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2494320896}
I20260709 13:16:19.973930 15240 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.984025 15444 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:19.996544 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 135669us
I20260709 13:16:19.991804 15223 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:19.993949 14783 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:20.023476 15387 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.024390 15387 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.027067 15387 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.027586 15239 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:20.029772 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.129452 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.129676 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.129944 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.130239 11512 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:20.130503 11511 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:16:20.130681 11516 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:20.031772 15407 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:20.131582 11519 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:20.033439 15406 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.036213 14775 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.040653 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.131803 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:20.133333 11514 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:20.041060 15370 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.134054 15370 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.136551 15370 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:20.004166 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 522948us
I20260709 13:16:20.047528 15450 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:20.140240 15450 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.142760 15450 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.048027 15362 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.143932 15362 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.055876 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:20.056803 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.147082 11444 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:20.057361 15357 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:20.151865 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.154655 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:20.061012 15365 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.197865 15365 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:20.065227 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:20.199888 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 134797us
I20260709 13:16:20.202055 11361 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:20.202517 15448 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:20.202926 15448 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.203248 15448 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:20.205821 15384 consensus_meta.cc:287] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.726s	user 0.000s	sys 0.014s
I20260709 13:16:20.067586 15223 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.068250 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:20.070152 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:20.207731 12063 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 5.
W20260709 13:16:20.207901 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 53386.7us
I20260709 13:16:20.070657 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:20.208770 12067 raft_consensus.cc:2368] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:20.208671 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:20.210049 15345 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:20.206715 15223 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.212776 15223 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.213305 15223 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:20.070910 15213 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:20.071627 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:20.072827 12069 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:20.073607 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:20.080341 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 688978us
W20260709 13:16:20.084721 15437 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.403s	user 0.003s	sys 0.000s
I20260709 13:16:20.086745 15222 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:20.094533 15232 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:20.095782 11485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.008s
I20260709 13:16:20.097339 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.097810 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.108170 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.115686 12060 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:20.116622 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:20.057691 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.156540 11268 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 8 to 9, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 9 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.131894 15407 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.162289 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.169533 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.171113 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.174304 14775 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:20.227105 14775 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.229172 14775 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:20.231451 11485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.011s	sys 0.008s
I20260709 13:16:20.231719 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:20.232280 11515 consensus_meta.cc:287] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.570s	user 0.017s	sys 0.003s
I20260709 13:16:20.232801 11515 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:20.234236 15448 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.214927 15223 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:16:20.235761 11437 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:20.239084 15407 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.239850 12062 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:20.240891 11442 raft_consensus.cc:2368] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10.
I20260709 13:16:20.241314 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:20.241815 11369 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.242304 15430 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:20.246145 15404 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.000s
I20260709 13:16:20.246625 15404 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.247771 11518 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:20.247541 15232 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.175242 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:20.185972 15388 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.595s	user 0.012s	sys 0.002s
I20260709 13:16:20.210479 15345 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.188189 15444 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:20.191469 15220 consensus_meta.cc:287] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.636s	user 0.016s	sys 0.003s
I20260709 13:16:20.191969 15362 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.193847 11510 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:20.194351 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.194702 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.196161 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.196506 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.196960 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.206457 15384 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:20.143922 11514 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:20.215850 12068 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:20.206676 15374 consensus_meta.cc:287] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.560s	user 0.005s	sys 0.010s
I20260709 13:16:20.225311 11444 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:16:20.223898 15213 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:20.187016 15368 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.003s
W20260709 13:16:20.256036 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 80916.1us
I20260709 13:16:20.242955 15430 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:20.257529 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 60100.1us
I20260709 13:16:20.258040 15405 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:20.253621 15345 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.259382 11335 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:20.260056 15357 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:20.260139 15358 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:16:20.262766 15325 consensus_meta.cc:287] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.588s	user 0.000s	sys 0.013s
I20260709 13:16:20.251091 15370 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.293448 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:20.258870 15418 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:20.301543 11368 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:20.312183 11366 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:20.298739 14783 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:20.318257 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 93598.1us
I20260709 13:16:20.304823 15357 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:20.302224 15374 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.324029 11484 leader_election.cc:400] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10.
I20260709 13:16:20.327100 11366 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:20.344049 11484 leader_election.cc:403] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:20.318153 15325 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:20.350443 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 141879us
I20260709 13:16:20.302724 15368 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.356339 15113 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:20.357093 15113 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.348366 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:20.343523 15223 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:20.317957 15362 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:20.323480 14638 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:20.329398 15357 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:20.325965 14775 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.324919 15213 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:20.369365 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.373456 11444 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:20.374903 11369 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
W20260709 13:16:20.376840 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54489.1us
I20260709 13:16:20.378026 11445 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:20.376605 11334 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:20.379925 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:20.380306 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.320325 15370 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.382719 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:20.387039 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 127809us
I20260709 13:16:20.387161 15430 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.388238 15113 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:20.389426 15442 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:20.401206 15374 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.411791 11268 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:20.413507 11268 catalog_manager.cc:5697] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.425158 15358 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:16:20.425652 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 51474.4us
I20260709 13:16:20.426412 15440 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:20.426554 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:20.427776 11334 leader_election.cc:400] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:20.428092 11334 leader_election.cc:403] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:16:20.429291 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.320452 14783 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.425814 15358 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:20.379164 15246 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.371940 15444 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:20.447649 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59797.4us
I20260709 13:16:20.445365 12068 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
W20260709 13:16:20.448232 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 81675.4us
I20260709 13:16:20.415010 11268 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 3 to 4, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 4 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.450510 11368 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:20.377494 15388 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10.
I20260709 13:16:20.453974 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:20.384478 15223 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:20.457247 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:20.458997 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 84159.7us
I20260709 13:16:20.459123 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.456162 15442 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:16:20.458091 15450 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:16:20.463263 11443 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:20.463634 15290 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.380825 14775 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.464296 15290 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.466399 15290 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.466609 14775 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.467885 15465 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:20.468274 15465 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:20.468809 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.469239 15465 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.469792 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.470542 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.471051 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.471355 11361 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:20.472236 11366 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:16:20.473099 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 86783.3us
I20260709 13:16:20.473591 15223 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.476195 15113 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:20.475983 11268 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 9, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 9 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.478516 15113 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:20.478914 15113 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:20.467571 15444 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.479992 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.481057 15254 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:20.481488 15254 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:20.478389 11268 catalog_manager.cc:5697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.470120 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.480947 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.472316 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.496024 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.496945 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:20.498332 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:20.499577 15254 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.500837 15254 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.441969 15448 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.450469 15404 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:20.502462 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:20.503929 15113 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.505266 11444 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:20.505069 15427 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:20.470498 15465 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:20.509217 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:20.510838 11363 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:20.511580 15440 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:20.511718 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 57865.7us
I20260709 13:16:20.512118 15440 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.514714 15427 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:20.515489 15463 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:20.506740 11268 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.505196 15113 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:20.517490 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 746) took 1015 ms. Trace:
I20260709 13:16:20.517864 12061 rpcz_store.cc:276] 0709 13:16:19.501954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.502086 (+   132us) service_pool.cc:224] Handling call
0709 13:16:20.517141 (+1015055us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b540034b554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:20.517301 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703569280}
I20260709 13:16:20.519994 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:20.520347 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.520710 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.520973 11511 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:20.521770 11519 raft_consensus.cc:2368] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7.
I20260709 13:16:20.522130 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.522508 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.523224 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.523583 11368 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:20.523696 15388 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:20.523798 15290 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:20.524350 15388 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:20.524816 15388 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:20.521347 11512 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:20.514919 15440 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.502574 15384 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:20.501900 15407 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:20.516626 11268 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 9 to 10, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 10 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.504526 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:20.530203 12065 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:20.524947 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:20.525259 15290 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:20.522354 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.531653 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.535105 11512 raft_consensus.cc:2410] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:16:20.532814 11518 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:20.449970 15374 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:20.536857 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.447465 15220 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:20.536868 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.539529 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:20.543753 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.525259 15298 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:20.501847 14783 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.441216 14638 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.515955 15463 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:20.531850 15384 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:20.552116 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 55270us
I20260709 13:16:20.552345 11511 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:20.552634 15450 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:20.549903 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 45436.1us
I20260709 13:16:20.550185 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.560540 11442 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:20.641573 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.536499 15374 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.605295 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:20.634712 11362 raft_consensus.cc:2410] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:16:20.584914 11361 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:20.586491 15384 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:20.620947 15220 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.588848 11369 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:20.559247 15440 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.712024 11437 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:20.590479 11519 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:20.536453 15358 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.558918 15407 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:20.566921 11370 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:20.597271 11268 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 8 to 10, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 10 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.537192 15370 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:20.647392 11439 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.027s	sys 0.060s
W20260709 13:16:20.723484 11439 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.027s	sys 0.060s
I20260709 13:16:20.647536 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.648067 15430 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:20.648872 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.651705 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:20.613446 15388 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.654402 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:20.656117 15325 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.656514 12065 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:20.652066 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:20.689723 15450 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.740723 11364 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.662122 11518 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:16:20.588485 15427 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:20.725911 11443 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:20.585947 15463 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:20.663239 12067 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:20.663410 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:20.668710 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:20.673656 12066 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:20.752756 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.688625 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:20.709015 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:20.667830 15465 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.715541 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:20.721467 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.689018 15368 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.709929 15232 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.741226 15254 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:20.665284 15246 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:20.755311 11440 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:20.746893 11268 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:20.758158 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 81606.4us
W20260709 13:16:20.758306 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 153161us
I20260709 13:16:20.758642 11511 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:20.746845 15450 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:20.764912 11518 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:20.765496 15092 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:20.766018 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:20.766943 11511 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:20.767947 15220 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.769443 11517 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:20.759241 15222 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.770311 15345 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.770380 11268 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:20.772183 11263 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56588 (request call id 237) took 18640 ms (client timeout 15000 ms). Trace:
I20260709 13:16:20.772625 15465 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.765543 15248 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:20.778254 11364 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:20.795578 11515 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:20.795614 15222 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.796238 11268 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 7, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 7 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:20.785035 14638 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:20.805893 15254 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:20.807235 11436 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:20.813068 15248 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:20.783833 15440 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:20.829407 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 140938us
W20260709 13:16:20.829105 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.051s	sys 0.101s
W20260709 13:16:20.835700 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.052s	sys 0.101s
W20260709 13:16:20.836021 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 538899us
I20260709 13:16:20.836897 15248 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.838217 15407 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:20.841773 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 75902.5us
I20260709 13:16:20.842608 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:20.844177 15298 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:20.845939 15388 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:20.850041 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.848322 12032 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:20.851859 15444 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.852728 15430 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:20.853793 15430 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:20.866621 11268 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 8, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.894838 11363 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:20.867374 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:20.855249 15430 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:20.831925 15374 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:20.907761 11511 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:16:20.904165 11268 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 9, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:20.901604 14638 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:20.814905 15418 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:20.913030 14638 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:20.917778 14638 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:20.919062 14638 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:20.897821 15092 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:20.903501 15430 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.928767 11333 leader_election.cc:400] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7.
W20260709 13:16:20.934759 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 67471.4us
I20260709 13:16:20.894817 15442 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:16:20.936712 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:20.939384 11268 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:20.941593 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:20.901491 15347 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:16:20.945798 11511 raft_consensus.cc:2410] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:16:20.947577 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:20.950959 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.930083 15374 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.932183 15430 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:20.938141 15368 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:20.947127 15365 consensus_meta.cc:287] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.749s	user 0.012s	sys 0.000s
I20260709 13:16:20.966858 11268 catalog_manager.cc:5697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.978338 11335 leader_election.cc:400] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
W20260709 13:16:20.952773 15357 consensus_meta.cc:287] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.002s
I20260709 13:16:20.952659 15444 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:20.997890 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:21.000834 11268 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 9, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 9 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.979223 15298 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:20.793310 11263 rpcz_store.cc:269] 0709 13:16:02.134116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.234149 (+100033us) service_pool.cc:224] Handling call
0709 13:16:20.760785 (+18526636us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:18.226889 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:18.227453 (+   564us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:18.227485 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:16:18.227503 (+    18us) tablet.cc:662] Decoding operations
0709 13:16:18.228491 (+   988us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:18.228566 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:18.228595 (+    29us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:16:18.229530 (+   935us) tablet.cc:701] Row locks acquired
0709 13:16:18.229549 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:16:18.229691 (+   142us) write_op.cc:270] Start()
0709 13:16:18.229801 (+   110us) write_op.cc:276] Timestamp: P: 1783602978229647 usec, L: 0
0709 13:16:18.229823 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:16:18.332287 (+102464us) log.cc:844] Serialized 4362 byte log entry
0709 13:16:18.672430 (+340143us) op_driver.cc:464] REPLICATION: finished
0709 13:16:19.247967 (+575537us) write_op.cc:301] APPLY: starting
0709 13:16:19.251436 (+  3469us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:19.358190 (+106754us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:19.358288 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:19.359274 (+   986us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:19.359566 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:16:19.359607 (+    41us) write_op.cc:312] APPLY: finished
0709 13:16:19.359877 (+   270us) log.cc:844] Serialized 83 byte log entry
0709 13:16:19.387306 (+ 27429us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:19.427714 (+ 40408us) write_op.cc:454] Released schema lock
0709 13:16:19.438470 (+ 10756us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:19.546020 (+107550us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:19.553961 (+  7941us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":55973,"mutex_wait_us":723183,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":528160,"apply.run_cpu_time_us":5947,"apply.run_wall_time_us":269709,"num_ops":11,"prepare.queue_time_us":145108,"prepare.run_cpu_time_us":17510,"prepare.run_wall_time_us":231467,"raft.queue_time_us":141508,"raft.run_cpu_time_us":13114,"raft.run_wall_time_us":304424,"replication_time_us":387328,"thread_start_us":416842,"threads_started":4,"wal-append.queue_time_us":104042,"wal-append.run_cpu_time_us":26487,"wal-append.run_wall_time_us":1966587}]]}
I20260709 13:16:21.010051 11511 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:21.013237 15450 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:20.981791 15368 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:20.997711 15444 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.035287 11268 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 8, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.979038 15374 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.040692 11513 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:16:21.048945 11511 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:20.952513 15092 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.046494 15444 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:21.046705 11513 pending_rounds.cc:85] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 9
I20260709 13:16:21.016844 15298 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.024200 15325 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:20.996313 15357 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.023557 15222 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.033298 15465 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.070982 15298 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:21.017802 15448 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 13:16:21.067523 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.068845 11513 pending_rounds.cc:107] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 10.10
W20260709 13:16:21.095337 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 149245us
I20260709 13:16:21.079432 11268 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:21.095690 11513 raft_consensus.cc:2889] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:21.068821 15087 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.100345 11512 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:21.101699 11516 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:21.084661 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 407892us
I20260709 13:16:21.102463 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2501) took 1061 ms. Trace:
I20260709 13:16:21.078850 15223 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:21.114427 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:21.123039 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.101653 15440 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.108855 15404 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.108115 11439 rpcz_store.cc:276] 0709 13:16:20.041085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.041210 (+   125us) service_pool.cc:224] Handling call
0709 13:16:20.041605 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:20.041840 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:20.747104 (+705264us) log.cc:844] Serialized 14 byte log entry
0709 13:16:21.102095 (+354991us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:21.102147 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:21.102200 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.102266 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.102399 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":722284,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":8001,"thread_start_us":971716,"threads_started":2}
I20260709 13:16:21.047751 15418 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:21.171923 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 74860.6us
I20260709 13:16:21.172600 15368 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.174283 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:21.174973 15092 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:21.176095 15483 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:16:21.123044 15087 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.180995 15374 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:21.181967 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 409025us
I20260709 13:16:21.182350 15232 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:16:21.201514 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56647.8us
I20260709 13:16:21.129637 11268 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 7, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.216056 12062 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:16:21.210024 11516 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:21.218500 12066 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:21.219058 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59649.4us
I20260709 13:16:21.220863 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:21.121328 15254 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:16:21.220978 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.229058 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:21.230580 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 715783us
I20260709 13:16:21.225948 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:21.238070 15480 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:16:21.240160 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.241014 12061 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:21.241582 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.241995 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.242951 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:21.243791 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.245144 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.246608 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.247834 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.249142 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.249758 12066 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:21.250744 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:21.252866 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:21.253664 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:21.254594 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.255810 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.256289 15213 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:21.256429 12065 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:21.256724 15213 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.257102 15213 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:21.251307 12066 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:21.258788 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:21.259732 15486 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:21.260174 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:21.260859 11441 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:21.262123 15476 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:21.262761 15476 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.263164 15476 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:21.263861 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:21.264554 11440 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 5.
I20260709 13:16:21.265596 15374 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.266137 15374 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.268668 15374 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.243093 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:21.277987 11437 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:21.280097 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.281385 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.282136 12064 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:21.282603 15478 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:21.282951 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.285073 15170 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:21.285704 15087 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:21.287827 12032 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.523s (SENT) [suppressed 26 similar messages]
I20260709 13:16:21.289116 12066 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:21.236008 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.183421 15223 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:21.281697 15474 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.560s	user 0.003s	sys 0.006s
I20260709 13:16:21.301421 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:21.276357 15325 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.304104 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:21.305183 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:21.305650 11367 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:21.306223 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:21.306988 12066 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:16:21.294517 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:21.295622 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:21.192416 15404 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.293462 15170 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:21.308017 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:21.309453 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:21.313939 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:21.312868 15087 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:21.316668 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.322309 11364 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:21.315491 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:21.235463 14638 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:16:21.315565 12061 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:21.310420 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.324558 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:21.327548 12064 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:21.299224 15450 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.293608 15407 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:21.302179 15498 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.331676 11361 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:21.332501 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:21.329780 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:21.334587 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:21.337999 11408 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:21.335718 11366 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:21.341948 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:21.322239 15248 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:21.299979 15092 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:16:21.330364 15232 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:21.354050 15170 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.355132 15170 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:21.346670 15357 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.356056 15357 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:21.356428 15357 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.356757 15357 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:21.358517 12067 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:21.338716 15325 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.371279 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 758) took 1152 ms. Trace:
I20260709 13:16:21.341603 15347 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.322000 15087 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:21.374606 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 148818us
W20260709 13:16:21.373673 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63354us
I20260709 13:16:21.351788 15220 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:21.378556 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:21.358950 15232 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.359423 15427 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:21.405851 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 67947.2us
I20260709 13:16:21.406778 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:21.407282 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.371780 12069 rpcz_store.cc:276] 0709 13:16:20.218660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.218824 (+   164us) service_pool.cc:224] Handling call
0709 13:16:20.219204 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:20.219821 (+   617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:20.219852 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:20.220389 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:16:20.563472 (+343083us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:20.563650 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.564843 (+  1193us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:21.281015 (+716172us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:21.362161 (+ 81146us) raft_consensus.cc:1685] finished
0709 13:16:21.362182 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.371198 (+  9016us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":765,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":107,"spinlock_wait_cycles":215866624,"thread_start_us":341811,"threads_started":1,"wal-append.queue_time_us":501034}
W20260709 13:16:21.414247 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 159722us
I20260709 13:16:21.425282 15450 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:21.351284 15222 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 13:16:21.417935 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 85540.7us
I20260709 13:16:21.384680 15232 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:21.461448 12064 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:21.414594 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:21.462911 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 175249us
I20260709 13:16:21.421296 15213 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.409755 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:21.413719 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.465379 11518 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:21.420179 11511 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:21.435897 15440 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:21.377445 15448 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.427709 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.385684 15427 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.352242 15499 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:16:21.424713 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:21.382594 15486 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:16:21.408762 11335 leader_election.cc:400] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:21.360683 15361 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.199s
I20260709 13:16:21.388553 11437 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:21.351284 15407 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.460788 11364 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:21.419911 15087 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.463150 11519 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:21.405162 15477 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:16:21.465418 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 141875us
W20260709 13:16:21.486783 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 80099.3us
W20260709 13:16:21.474743 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50095.1us
I20260709 13:16:21.490351 12062 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:21.380709 15220 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:21.374101 15325 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.440531 15365 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:21.478523 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.360144 15498 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.483390 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.490794 11517 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:21.464191 15487 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:21.494829 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.476161 15427 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:21.496731 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 939) took 1022 ms. Trace:
I20260709 13:16:21.494063 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:21.502504 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:21.408156 15347 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.487144 15477 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:21.517191 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 130324us
I20260709 13:16:21.511298 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:21.512364 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.505609 11369 rpcz_store.cc:276] 0709 13:16:20.467788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.468551 (+   763us) service_pool.cc:224] Handling call
0709 13:16:21.490188 (+1021637us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400165654. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:21.490313 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274210176}
I20260709 13:16:21.475303 15442 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:16:21.515525 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:21.522774 11518 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:21.515278 15213 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.533720 11439 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:21.542258 11443 raft_consensus.cc:2368] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:16:21.542356 12061 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:21.524343 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.543510 11441 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:21.525424 15345 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.500643 15365 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:21.527302 15503 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:16:21.529492 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.530289 15427 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.496578 15220 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:21.518213 11445 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:21.536800 11442 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:21.536882 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.537448 15511 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:21.532318 15477 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:21.539813 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1718) took 1081 ms. Trace:
I20260709 13:16:21.539996 15223 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.540601 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:21.567145 11515 rpcz_store.cc:276] 0709 13:16:20.458685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.458830 (+   145us) service_pool.cc:224] Handling call
0709 13:16:20.479545 (+ 20715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:20.802677 (+323132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.539578 (+736901us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54000663d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:21.539731 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726309120}
I20260709 13:16:21.568707 11484 leader_election.cc:400] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:16:21.571118 11484 leader_election.cc:403] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:16:21.554270 15222 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:16:21.570284 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 92157.3us
I20260709 13:16:21.489239 15444 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.526839 15509 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:21.548476 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 65212.5us
I20260709 13:16:21.560979 11511 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:21.532850 15450 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.547369 15113 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:21.545841 15478 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.574538 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.580035 11517 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
W20260709 13:16:21.545710 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 67274.8us
I20260709 13:16:21.575515 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:21.563755 15347 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.580720 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 942) took 1108 ms. Trace:
I20260709 13:16:21.582780 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.582109 15507 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:21.564510 15511 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:21.588076 15504 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:21.592614 15427 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.572522 15514 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.572467 15087 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:21.582549 11362 rpcz_store.cc:276] 0709 13:16:20.468772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.479743 (+ 10971us) service_pool.cc:224] Handling call
0709 13:16:21.572299 (+1092556us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400259754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:21.577387 (+  5088us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353863552}
I20260709 13:16:21.585273 15507 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:21.578562 15345 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:16:21.576438 15465 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:16:21.601429 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 89150.5us
I20260709 13:16:21.574275 15222 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:21.599327 11333 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:21.611331 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 121958us
I20260709 13:16:21.611608 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.580720 15361 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:21.624634 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.600064 15504 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.644841 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:21.647157 15298 consensus_meta.cc:287] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.002s
I20260709 13:16:21.648643 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 805) took 1008 ms. Trace:
I20260709 13:16:21.638068 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:21.610605 15087 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.635677 15476 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:21.655781 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 62064.2us
I20260709 13:16:21.608198 14638 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:21.649029 11436 rpcz_store.cc:276] 0709 13:16:20.640467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.640606 (+   139us) service_pool.cc:224] Handling call
0709 13:16:21.648456 (+1007850us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b5400438ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:21.648585 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79210880}
I20260709 13:16:21.647563 15223 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:21.633775 15222 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:21.633396 15361 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:21.634264 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 78449.6us
I20260709 13:16:21.629081 15113 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.582563 15365 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.664781 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:21.627267 15465 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:21.643973 15504 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:21.696349 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 84810.2us
I20260709 13:16:21.656947 15087 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:21.657552 15384 consensus_meta.cc:287] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.911s	user 0.012s	sys 0.000s
I20260709 13:16:21.643575 15427 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:16:21.706940 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.665796 15478 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.659379 14638 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:21.713835 15522 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:16:21.680238 15520 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.647594 15298 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.718367 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.679809 15516 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.718510 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:21.679845 15512 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:21.727278 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.688158 15498 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.703123 15213 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:21.729838 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 91866.7us
I20260709 13:16:21.731021 11515 raft_consensus.cc:2410] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:16:21.648260 15514 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.709904 15407 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:21.738526 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.669049 15223 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.729863 15418 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:16:21.766957 15405 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:21.773056 15405 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:21.736296 15507 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.773499 15405 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:21.774107 15223 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.726389 15325 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.753543 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:21.775934 11362 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:21.768903 15513 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:21.776525 15248 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:21.776945 15248 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:21.776682 15513 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.737607 15516 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.777858 15248 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.779136 15248 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:21.763592 15465 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.717948 15511 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.780469 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 878) took 2354 ms. Trace:
I20260709 13:16:21.737973 15478 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:21.757244 15361 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.783783 12032 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:21.755723 11370 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:16:21.747123 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.767153 15370 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 13:16:21.750970 15407 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.769357 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:21.783125 15514 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.727839 14638 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:21.772159 11438 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:16:21.932528 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.915951 11515 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:21.923213 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:21.857666 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.790195 11362 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:21.713024 15384 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.860217 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:21.756927 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.945112 11408 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:21.915731 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 143501us
I20260709 13:16:21.784725 11513 rpcz_store.cc:276] 0709 13:16:19.426111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.592707 (+166596us) service_pool.cc:224] Handling call
0709 13:16:19.592966 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.106199 (+1513233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.780224 (+674025us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002a80d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:21.780401 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3235733248}
I20260709 13:16:21.945938 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:21.950755 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.951766 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.913370 15525 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:21.917135 15357 consensus_meta.cc:287] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.001s
I20260709 13:16:21.955104 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:21.753058 11364 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:21.950618 11333 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:21.955935 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:21.960773 11517 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:21.957162 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 247598us
I20260709 13:16:21.764539 15476 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:21.915019 15499 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.757160 15508 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:16:21.758245 15518 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:16:21.917553 15325 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.969746 11335 leader_election.cc:400] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
I20260709 13:16:21.769016 15448 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.970930 11335 leader_election.cc:403] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:21.757592 15444 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.744062 15113 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.207s
I20260709 13:16:21.754338 15298 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:21.732722 15520 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:21.979723 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.980036 15516 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.980599 11513 raft_consensus.cc:2410] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:16:21.981137 15520 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:21.855351 15213 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:21.999660 15499 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.748116 15512 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.770880 15427 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:21.744853 15486 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.224s
W20260709 13:16:21.970607 15430 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.994s	user 0.009s	sys 0.002s
I20260709 13:16:22.025682 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:21.971706 15503 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:21.969837 15345 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.947367 15170 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:22.048735 11513 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:22.030637 15325 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.970731 15440 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:21.970048 15508 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:21.974742 15223 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:22.023612 15465 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:22.119302 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:22.021894 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 252680us
I20260709 13:16:22.048039 15362 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:22.033135 15514 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:22.026175 15368 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:22.029062 15516 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:21.955264 15407 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:21.969142 15357 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.073405 11511 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
W20260709 13:16:22.082803 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 136930us
W20260709 13:16:22.133960 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 164362us
I20260709 13:16:22.135321 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:21.976063 15478 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:22.090052 11517 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:22.070310 15520 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
W20260709 13:16:22.111485 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 251305us
I20260709 13:16:22.114301 11437 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:22.111912 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.000226 15448 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.139930 12062 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 13:16:22.140799 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:22.141324 15531 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:22.141603 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.141839 15535 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:22.142256 15535 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.142576 15535 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:22.145066 14638 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.137647 15478 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:22.147090 14638 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.148244 15478 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:22.133708 15213 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:22.152179 15232 consensus_meta.cc:287] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.667s	user 0.008s	sys 0.001s
I20260709 13:16:22.083094 15430 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.153296 15222 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.123323 15223 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.155767 15508 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.156734 12063 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:22.151495 15370 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:16:22.125547 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:22.123162 15477 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 13:16:22.083591 15513 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.074079 15427 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:22.122025 15220 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.002s
I20260709 13:16:22.152642 15232 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.141353 11441 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:22.138761 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:22.114807 15440 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.160498 11438 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:22.091675 15345 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.155180 15430 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.156848 15508 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:22.167649 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:22.172032 15475 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:22.172627 15220 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.172923 15362 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:22.174950 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.181187 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:22.133917 15516 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.156898 15223 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:22.169274 15463 consensus_meta.cc:287] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.382s	user 0.006s	sys 0.007s
I20260709 13:16:22.163480 15514 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:22.173529 15504 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.174674 15232 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.182397 15440 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:22.131882 15368 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:22.203783 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 179206us
W20260709 13:16:22.193344 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 54656us
I20260709 13:16:22.208905 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1721) took 1688 ms. Trace:
I20260709 13:16:22.210089 15220 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.227548 11516 rpcz_store.cc:276] 0709 13:16:20.520316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.524715 (+  4399us) service_pool.cc:224] Handling call
0709 13:16:22.208724 (+1684009us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b5400109854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:22.208845 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1312086912}
I20260709 13:16:22.204780 15498 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:22.229112 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.198055 15362 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:22.231364 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:22.198729 15170 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.255625 15404 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.255888 15362 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.211376 15463 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.258188 15407 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.258669 15362 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.258821 15463 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.259682 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.261303 11369 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:22.262830 15222 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.264431 15357 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.164891 15477 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.268965 11365 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:22.269688 12061 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:22.186530 15520 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:16:22.286676 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.214694 15513 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:22.219961 15087 consensus_meta.cc:287] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.502s	user 0.002s	sys 0.006s
W20260709 13:16:22.250598 15523 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:16:22.250886 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:22.251204 15540 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:16:22.251799 15529 log.cc:927] Time spent T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:16:22.252063 11516 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:22.252858 15368 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.206681 15384 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.214217 15113 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:22.252676 15534 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:16:22.245136 11441 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:22.165339 15531 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.288710 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.309652 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:22.252795 15499 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.309727 11366 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:22.292641 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 71893.3us
I20260709 13:16:22.231043 15542 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:22.223479 15418 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:16:22.317874 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2240) took 1795 ms. Trace:
I20260709 13:16:22.318214 11512 rpcz_store.cc:276] 0709 13:16:20.522028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.793854 (+271826us) service_pool.cc:224] Handling call
0709 13:16:20.819610 (+ 25756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.105959 (+286349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.317653 (+1211694us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b540031f954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:22.317806 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642169216}
I20260709 13:16:22.307822 11438 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:16:22.288750 15405 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.000s
I20260709 13:16:22.256590 15516 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.329702 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:22.330298 11512 raft_consensus.cc:2376] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10. Re-sending same reply.
I20260709 13:16:22.331112 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:22.317461 15512 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:22.315021 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:22.332083 11365 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 945) took 1862 ms (client timeout 1951 ms). Trace:
I20260709 13:16:22.221467 15514 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.334131 15514 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.334642 15514 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.285662 15498 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:22.336119 15498 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.336470 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:22.337028 11510 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:22.197911 15475 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.300104 15220 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:22.335088 15514 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.263052 15535 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.326297 11516 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:16:22.342962 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:22.343439 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1199) took 1129 ms. Trace:
I20260709 13:16:22.317467 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:22.343734 11516 raft_consensus.cc:2410] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:16:22.344816 11516 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:22.300971 15513 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.243850 15507 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:16:22.326670 15405 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.353152 12063 rpcz_store.cc:276] 0709 13:16:21.214049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.214355 (+   306us) service_pool.cc:224] Handling call
0709 13:16:22.343179 (+1128824us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54005e29d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:22.343330 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2131536256}
I20260709 13:16:22.328492 15418 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:22.295840 15477 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:22.341715 15475 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.347026 11512 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:22.322038 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:22.346757 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.288146 15170 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.353011 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:22.332520 11365 rpcz_store.cc:269] 0709 13:16:20.469765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.469871 (+   106us) service_pool.cc:224] Handling call
0709 13:16:22.331919 (+1862048us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400249d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:22.332023 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4107519360}
I20260709 13:16:22.354827 12065 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:22.355582 11408 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:22.356041 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2509) took 1820 ms. Trace:
I20260709 13:16:22.366780 11441 raft_consensus.cc:2368] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6.
I20260709 13:16:22.354374 11366 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:22.309075 15540 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:22.326234 15345 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:22.300634 15087 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.370215 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.358968 15407 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:22.373780 11408 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:22.366816 11444 rpcz_store.cc:276] 0709 13:16:20.535725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.535868 (+   143us) service_pool.cc:224] Handling call
0709 13:16:20.549875 (+ 14007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:21.540394 (+990519us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b2c0016a428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bacae291 00007f35bacc90ca 00007f35bc9f74ad 00007f35bc9fde1b 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:21.540534 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:21.540561 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:21.541096 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:16:21.703286 (+162190us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:21.703321 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.703416 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:22.348849 (+645433us) raft_consensus.cc:1685] finished
0709 13:16:22.348879 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:22.355977 (+  7098us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22123,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":12904,"spinlock_wait_cycles":2953472,"thread_start_us":161804,"threads_started":1,"wal-append.queue_time_us":383496}
I20260709 13:16:22.314520 15113 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.323000 15357 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 13:16:22.382407 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.387717 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.388368 11441 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:22.388651 15170 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.389217 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.378680 15407 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:22.390600 15407 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:22.391273 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:22.392581 11442 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:22.393721 11442 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:22.394717 15540 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.396044 15504 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.377501 15512 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:22.396952 15512 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.397277 15512 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:22.400238 15516 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:22.400875 15516 consensus_queue.cc:1243] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 34a42aa6aa0a4778a532c6433d70b981 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:16:22.401384 15113 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.346698 15542 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.403699 15405 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.403999 11442 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:22.407804 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:22.414014 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.414625 11444 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
W20260709 13:16:22.424656 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.445262 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.446178 11366 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:22.484905 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:22.501154 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.370339 15440 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.506912 15504 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:22.508255 15440 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.509027 11367 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:16:22.509455 15525 consensus_meta.cc:287] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.005s
I20260709 13:16:22.509986 15525 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:22.391176 15478 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.511350 12067 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:22.512439 15478 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.518296 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.518989 11443 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:22.520223 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.506903 15542 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:22.528028 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.529407 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:22.530083 11444 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:22.530795 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:22.536528 15514 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.538187 15513 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.540361 15535 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.386761 15223 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:22.541329 15223 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:22.542302 15223 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.543641 15427 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.385610 15222 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.166s
I20260709 13:16:22.404106 15220 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.149s
I20260709 13:16:22.563341 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.564781 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.565419 11443 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
W20260709 13:16:22.569743 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 84933.5us
I20260709 13:16:22.578356 11512 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:22.593492 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:22.531265 11444 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:22.581883 11513 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:16:22.570864 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:22.617216 11510 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:22.618196 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:22.618795 11510 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:22.620038 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.621160 11443 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 14: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:22.622321 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.623272 11443 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:22.623742 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:22.624809 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:22.628110 15531 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:22.630427 15465 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.506s	user 0.003s	sys 0.006s
W20260709 13:16:22.647763 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 76953.5us
I20260709 13:16:22.648679 11483 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:22.649926 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:22.651379 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:22.660619 15507 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:22.676229 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.694947 11443 raft_consensus.cc:2410] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:16:22.697789 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:22.698554 11510 raft_consensus.cc:2368] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:22.699625 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2234) took 2179 ms. Trace:
W20260709 13:16:22.699803 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 292085us
I20260709 13:16:22.702065 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:22.704527 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.705240 11369 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:22.705842 11444 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:22.705965 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:22.719389 11513 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:22.722481 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2514) took 1232 ms. Trace:
I20260709 13:16:22.722872 11439 rpcz_store.cc:276] 0709 13:16:21.489889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.490792 (+   903us) service_pool.cc:224] Handling call
0709 13:16:22.722328 (+1231536us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b54003e53d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:22.722426 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63195904}
I20260709 13:16:22.667613 15504 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:22.738170 11443 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:16:22.738636 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 427593us
I20260709 13:16:22.700055 11514 rpcz_store.cc:276] 0709 13:16:20.520262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.540576 (+ 20314us) service_pool.cc:224] Handling call
0709 13:16:20.556444 (+ 15868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:20.758808 (+202364us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:20.765494 (+  6686us) log.cc:844] Serialized 14 byte log entry
0709 13:16:21.209452 (+443958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:21.209587 (+   135us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:21.209637 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.699427 (+1489790us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540014d2d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:22.699467 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:22.699575 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2055,"prepare.queue_time_us":45116,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":32222,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5318,"spinlock_wait_cycles":401119104,"thread_start_us":441353,"threads_started":2,"wal-append.queue_time_us":414790,"wal-append.run_cpu_time_us":1124,"wal-append.run_wall_time_us":1002668}
I20260709 13:16:22.745065 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.745532 11513 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:16:22.745821 11366 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:22.697129 15542 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:22.749511 11439 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:16:22.711755 15220 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:22.711410 15535 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:22.756865 11513 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:22.722021 15222 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:16:22.758208 11513 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:16:22.761256 15407 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:22.762509 11442 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:22.763721 11442 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:22.765501 11442 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:22.766610 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1820) took 1471 ms. Trace:
I20260709 13:16:22.766983 11370 rpcz_store.cc:276] 0709 13:16:21.295590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.295702 (+   112us) service_pool.cc:224] Handling call
0709 13:16:21.304358 (+  8656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.914796 (+610438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.766389 (+851593us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540035b454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:22.766528 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037278592}
I20260709 13:16:22.768014 11365 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:16:22.768278 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.768965 11370 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:22.769047 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.769778 15448 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:22.770169 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:22.772066 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.775702 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.744611 15513 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:22.787117 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.776836 11438 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:22.754051 15535 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.751884 15504 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:22.776515 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:22.813153 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.046s	sys 0.092s
W20260709 13:16:22.814368 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.046s	sys 0.092s
I20260709 13:16:22.844026 12063 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:22.845285 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.845453 15087 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:22.845978 12063 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
W20260709 13:16:22.851910 11483 leader_election.cc:336] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 1.918s (SENT)
I20260709 13:16:22.852487 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:22.854364 15512 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.806710 15535 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:22.856714 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:22.857534 12066 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:22.858314 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.858539 15430 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:22.858904 15430 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:22.859076 12062 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:22.860535 15430 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:22.856822 11518 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1729) took 1326 ms (client timeout 1654 ms). Trace:
W20260709 13:16:22.862340 11518 rpcz_store.cc:269] 0709 13:16:21.530330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.536601 (+  6271us) service_pool.cc:224] Handling call
0709 13:16:22.856604 (+1320003us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b54000e8154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:22.856760 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1203729280}
I20260709 13:16:22.865429 15549 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:22.867925 15512 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:22.889156 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:22.871132 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.872222 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:22.872829 11483 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 1.928s (SENT)
I20260709 13:16:22.884339 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.788074 15542 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:22.892473 12069 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:22.892048 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:22.893827 11516 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.015s	sys 0.012s
W20260709 13:16:22.907121 11516 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.015s	sys 0.012s
I20260709 13:16:22.894245 15513 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:22.908598 15513 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:22.894646 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.899376 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:22.900575 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:22.902585 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:22.912921 11438 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:22.912933 11442 raft_consensus.cc:2368] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:22.913552 11445 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:22.905872 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.917143 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.917322 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.918124 12067 raft_consensus.cc:2410] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:16:22.920984 12063 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:22.921415 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.921630 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.921193 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:22.872000 15430 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:22.924786 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.930740 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:22.923580 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:22.927620 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:22.928436 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:22.928611 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 183938us
I20260709 13:16:22.933043 11369 raft_consensus.cc:2368] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9.
I20260709 13:16:22.885012 15536 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.944747 11365 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:22.940788 12062 raft_consensus.cc:2368] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:16:22.946863 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:22.951001 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.951869 12069 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:22.954360 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:22.955034 12063 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:22.955266 11370 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 12. Ops: [10.10-10.10]
I20260709 13:16:22.927654 15486 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:16:22.958884 11369 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:22.959842 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.939270 15476 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:22.964502 12066 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:22.964819 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:22.965894 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 93093.7us
W20260709 13:16:22.967511 11483 leader_election.cc:336] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 1.934s (SENT)
I20260709 13:16:22.967918 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:22.970634 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.975196 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:22.963105 15504 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.961022 15407 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:22.944337 15540 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:16:22.961512 15478 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:22.986572 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2251) took 1328 ms. Trace:
I20260709 13:16:22.973847 15542 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:22.977983 15486 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:22.995975 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:22.949047 15536 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.011583 11445 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:23.012626 12066 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:22.999989 11512 rpcz_store.cc:276] 0709 13:16:21.657827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.625599 (+967772us) service_pool.cc:224] Handling call
0709 13:16:22.625883 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:22.626069 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:22.806425 (+180356us) log.cc:844] Serialized 14 byte log entry
0709 13:16:22.985043 (+178618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:22.985196 (+   153us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:22.985451 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.986161 (+   710us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:22.986510 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":230056,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":25,"thread_start_us":337485,"threads_started":2}
W20260709 13:16:22.995217 15532 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 0.924s	user 0.000s	sys 0.015s
I20260709 13:16:23.020918 15516 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.021476 15516 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.022969 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 761) took 1779 ms. Trace:
I20260709 13:16:23.027887 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:23.029213 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:23.031354 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.034929 12063 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:23.023337 12065 rpcz_store.cc:276] 0709 13:16:21.243203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.257113 (+ 13910us) service_pool.cc:224] Handling call
0709 13:16:22.348065 (+1090952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:22.366399 (+ 18334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.022771 (+656372us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54005e29d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:23.022915 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2508284672}
W20260709 13:16:23.039549 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.011600 15549 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:22.969969 15476 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.055473 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.036154 15442 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:23.057550 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 90073us
I20260709 13:16:23.059742 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.058109 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.056149 11363 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:23.044247 15427 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:23.044643 15540 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:23.044117 15465 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:23.065526 15465 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.069454 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.069722 15465 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:23.070261 11367 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:23.075479 15478 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:23.079730 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.106598 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:23.050187 15222 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:16:23.116526 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.118351 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.112787 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.122169 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.122648 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.122794 11370 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 5.
I20260709 13:16:23.121258 11367 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:23.070089 15516 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:23.060629 15508 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 13:16:23.128192 15562 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:16:23.129208 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.130357 15511 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.131145 11483 leader_election.cc:400] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8.
I20260709 13:16:23.086834 15512 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.142668 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2250) took 1485 ms. Trace:
I20260709 13:16:23.143172 11516 rpcz_store.cc:276] 0709 13:16:21.657564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.345553 (+687989us) service_pool.cc:224] Handling call
0709 13:16:22.345846 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:22.346057 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:22.907506 (+561449us) log.cc:844] Serialized 14 byte log entry
0709 13:16:23.142182 (+234676us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.142233 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:23.142276 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.142502 (+   226us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.142610 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":509866,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"thread_start_us":795298,"threads_started":2}
I20260709 13:16:23.157122 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.158546 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.062681 15442 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.160708 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.161203 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 902) took 1770 ms. Trace:
I20260709 13:16:23.157764 11369 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:23.163250 11361 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:23.161559 11515 rpcz_store.cc:276] 0709 13:16:21.390599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.983549 (+592950us) service_pool.cc:224] Handling call
0709 13:16:22.065290 (+ 81741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:22.377952 (+312662us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b180004ee20. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35bc5a64bc 00007f35b98f448b 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35bc9f6060 00007f35bc9f6bfb 00007f35bca0164b
0709 13:16:22.385834 (+  7882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.385863 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.660484 (+274621us) log.cc:844] Serialized 26 byte log entry
0709 13:16:22.872772 (+212288us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:22.872804 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.872874 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.131641 (+258767us) raft_consensus.cc:1685] finished
0709 13:16:23.131663 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.161146 (+ 29483us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26120,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":19897,"spinlock_wait_cycles":10158848,"thread_start_us":264733,"threads_started":2,"wal-append.queue_time_us":312217}
I20260709 13:16:23.130586 15361 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:23.160530 11483 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.128201 15092 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:23.174258 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:23.192718 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 248526us
I20260709 13:16:23.197902 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:23.198629 12062 raft_consensus.cc:2368] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:23.199276 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.200031 11367 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:23.156432 15544 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:23.196774 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.201467 11361 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:23.206990 12065 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:23.214140 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.216759 12065 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:23.220291 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:23.220908 12065 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.223583 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.224277 12063 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:23.225275 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.156126 15407 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:16:23.203688 15544 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.193987 15554 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.205745 15531 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:23.250679 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:23.258759 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.272675 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:23.283317 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.284040 12063 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:23.284910 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.165693 15486 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.259624 15442 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:23.260459 15544 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.272516 15254 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:23.302381 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:23.288450 15478 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.280169 15531 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:23.300163 15427 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:23.298849 15442 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:23.278718 15465 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:16:23.266067 15554 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:23.329346 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b1906ab37ba541f493936685a808c8a9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:23.330144 11253 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 56423.3us
W20260709 13:16:23.333350 15560 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.357s	user 0.002s	sys 0.000s
I20260709 13:16:23.337657 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 906) took 1031 ms. Trace:
I20260709 13:16:23.301266 12065 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:23.286473 15498 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:16:23.338055 11514 rpcz_store.cc:276] 0709 13:16:22.305989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.750838 (+444849us) service_pool.cc:224] Handling call
0709 13:16:22.754977 (+  4139us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:22.774631 (+ 19654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.774663 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.790899 (+ 16236us) log.cc:844] Serialized 26 byte log entry
0709 13:16:23.193850 (+402951us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:23.193890 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.193961 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.337322 (+143361us) raft_consensus.cc:1685] finished
0709 13:16:23.337347 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.337592 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33009,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":128,"thread_start_us":274070,"threads_started":1,"wal-append.queue_time_us":415270}
I20260709 13:16:23.342409 15427 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:23.323455 15476 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:23.351267 15254 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:23.351802 15254 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:23.352319 15567 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:16:23.352692 15254 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.353317 15498 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:23.353806 15498 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:23.354251 15254 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:23.319530 15478 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.362957 15478 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:23.369009 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:23.369427 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:23.369987 11442 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.326516 15442 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:23.377620 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:23.378101 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:23.378284 11445 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:23.383448 15554 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:23.388787 15442 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:23.391891 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.393148 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.354676 15498 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.396176 11363 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:23.397471 11370 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:23.397854 11370 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:23.398680 11370 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:23.402722 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.405840 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.406719 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:23.408258 11370 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:23.396440 15498 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.414346 11362 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:23.415849 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.416548 11484 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.400666 15223 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.417533 15223 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:23.418521 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.419236 12063 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:16:23.387233 15535 consensus_meta.cc:287] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.531s	user 0.005s	sys 0.004s
W20260709 13:16:23.420840 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 476180us
I20260709 13:16:23.424713 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:23.425060 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 318512us
I20260709 13:16:23.425988 15503 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:23.426575 15503 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:23.427712 15503 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.399137 15544 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.433909 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2353) took 1036 ms. Trace:
W20260709 13:16:23.439618 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 391612us
I20260709 13:16:23.409672 15509 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:23.447805 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:23.442246 11444 rpcz_store.cc:276] 0709 13:16:22.397498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.722795 (+325297us) service_pool.cc:224] Handling call
0709 13:16:22.723108 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:22.739964 (+ 16856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.740004 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.747409 (+  7405us) log.cc:844] Serialized 26 byte log entry
0709 13:16:22.987080 (+239671us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:23.042599 (+ 55519us) log.cc:844] Serialized 14 byte log entry
0709 13:16:23.385595 (+342996us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b18003ef380. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:23.420466 (+ 34871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.420542 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.420581 (+    39us) raft_consensus.cc:1685] finished
0709 13:16:23.433586 (+ 13005us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.433791 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44416,"prepare.queue_time_us":55474,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2026,"raft.queue_time_us":45102,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":6443,"spinlock_wait_cycles":28890368,"thread_start_us":515288,"threads_started":2,"wal-append.queue_time_us":273348}
I20260709 13:16:23.443297 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.443436 11409 leader_election.cc:400] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
W20260709 13:16:23.447238 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:23.451370 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:23.451900 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.452301 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 816) took 1319 ms. Trace:
W20260709 13:16:23.452348 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.453864 11443 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:23.453917 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.454751 11362 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:23.451116 11409 leader_election.cc:403] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:16:23.452776 11440 rpcz_store.cc:276] 0709 13:16:22.133198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.140101 (+  6903us) service_pool.cc:224] Handling call
0709 13:16:22.140364 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:22.192632 (+ 52268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.192663 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:22.192702 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:22.192740 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.452046 (+1259306us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b54003142d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:23.452094 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.452240 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119751936}
I20260709 13:16:23.457942 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2361) took 1057 ms. Trace:
I20260709 13:16:23.458076 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.458669 11362 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:23.454790 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.458300 11443 rpcz_store.cc:276] 0709 13:16:22.400026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.750328 (+350302us) service_pool.cc:224] Handling call
0709 13:16:22.750601 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.453730 (+703129us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b54003e2720. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc975dc0 00007f35bca001c0 00007f35bca01231 00007f35bc9fdd8f 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c
0709 13:16:23.457716 (+  3986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.457882 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17555712}
I20260709 13:16:23.459631 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.433511 15506 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:23.460167 11362 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:23.468116 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.469084 11363 raft_consensus.cc:2410] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:23.466348 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.472642 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.470606 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.478657 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.479684 15448 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:23.480096 15448 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:23.480931 15448 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.451517 15509 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.486447 15506 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:23.487687 15325 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:23.491984 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.493091 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1040) took 1359 ms. Trace:
I20260709 13:16:23.406703 15531 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.461549 15442 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.460582 15362 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.496905 11362 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:23.496711 12061 rpcz_store.cc:276] 0709 13:16:22.134098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.140854 (+  6756us) service_pool.cc:224] Handling call
0709 13:16:22.141116 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:22.270096 (+128980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.492831 (+1222735us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400658e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:23.493028 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295156608}
I20260709 13:16:23.498059 15232 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:23.442800 15535 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:23.497334 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 52359.6us
I20260709 13:16:23.534135 12034 leader_election.cc:400] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:23.543875 12034 leader_election.cc:403] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:16:23.534943 11361 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:16:23.536329 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:23.499083 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.542256 11363 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:16:23.545467 11363 pending_rounds.cc:85] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 9
I20260709 13:16:23.537546 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:23.545727 11363 pending_rounds.cc:107] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:16:23.546092 11363 raft_consensus.cc:2889] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:23.545626 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:23.495455 15509 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.547220 15325 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:23.547750 15325 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 15 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:23.496052 15506 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.556425 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:23.549688 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50662.5us
I20260709 13:16:23.557278 12064 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:23.475576 15511 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:16:23.451248 15578 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
W20260709 13:16:23.552023 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 135550us
I20260709 13:16:23.475764 15478 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:16:23.561659 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.496534 15572 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:23.540202 15232 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.550429 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:23.567617 11361 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:23.566502 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.573654 11362 raft_consensus.cc:2410] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:16:23.496125 12066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:23.559319 15506 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:23.574772 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.542831 15362 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.577165 12066 raft_consensus.cc:606] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:23.559514 15577 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:16:23.581298 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.580265 12066 raft_consensus.cc:614] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:23.586267 11408 leader_election.cc:400] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:23.585573 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:23.588454 11443 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:23.567226 15572 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.578176 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:23.548571 15325 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.590741 11367 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:23.590368 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.689864 11362 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:23.539965 15442 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.573931 15232 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:23.684927 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:23.685672 15092 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.687148 11364 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 5.
I20260709 13:16:23.576907 15541 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:23.691486 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:23.703616 11443 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
W20260709 13:16:23.685411 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 90503.8us
I20260709 13:16:23.583600 15430 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:23.589887 15427 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:23.581524 15465 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:23.705228 11510 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:23.696171 11333 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:23.716320 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.708563 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:23.596971 15577 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:23.688858 15512 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:23.604642 15572 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:23.723626 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 186140us
I20260709 13:16:23.731278 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:23.744843 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.582263 15407 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 13:16:23.722556 15465 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.750360 11442 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:23.705977 15541 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:23.713661 15427 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:23.762322 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.742928 15544 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:23.728888 15503 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.705556 15092 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:23.725344 15361 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:23.780316 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.588967 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:23.787691 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 209581us
I20260709 13:16:23.793677 15503 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.794273 15362 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:23.748903 15430 consensus_queue.cc:1059] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:16:23.798947 12068 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:23.799539 15213 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:16:23.800177 15591 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:16:23.800491 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:23.801017 15588 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:23.767724 15465 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:23.751356 15595 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:23.815196 11362 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.775830 15427 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:23.768981 15531 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.821096 11442 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:23.821458 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 319704us
I20260709 13:16:23.794430 15442 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:23.776576 15589 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.758311 15581 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
W20260709 13:16:23.824086 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 235190us
W20260709 13:16:23.877501 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 169966us
I20260709 13:16:23.925238 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:23.921211 11370 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:16:23.944613 11511 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:23.924517 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:23.945322 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.945995 11370 raft_consensus.cc:2368] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9.
I20260709 13:16:23.904277 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.940802 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:23.866707 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 271049us
W20260709 13:16:23.949805 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 206461us
W20260709 13:16:23.936637 15579 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 13:16:23.965939 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:23.966492 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.966686 11518 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:16:23.864024 15583 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
W20260709 13:16:23.932530 15254 consensus_meta.cc:287] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.000s
I20260709 13:16:23.824723 15361 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:23.824064 15504 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:23.832741 15407 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:23.974473 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 440868us
I20260709 13:16:23.975054 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:23.980195 11512 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:23.975193 11513 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:16:23.844722 15511 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:23.978938 11516 pending_rounds.cc:85] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 5
I20260709 13:16:23.980976 11516 pending_rounds.cc:107] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:16:23.888845 15223 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:23.863756 15508 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:23.941972 15595 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:23.886868 15594 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:16:23.947816 15362 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:23.981237 11516 raft_consensus.cc:2889] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:23.985457 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:23.988723 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45975.1us
I20260709 13:16:23.990092 11438 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:23.980736 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:23.950335 15605 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:23.999366 11362 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:23.977334 15430 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.002652 15430 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.976521 15232 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.000218 15508 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.004916 15232 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:23.989212 15498 consensus_meta.cc:287] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.000s
I20260709 13:16:24.005230 15508 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.005529 15605 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.000779 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:23.981724 15486 raft_consensus.cc:3055] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:16:23.993013 15590 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:16:24.018585 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.027354 11511 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:24.028488 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:23.980134 15504 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:23.993988 15604 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.030968 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:24.029196 11511 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:23.948069 15541 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.032608 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:24.033315 11511 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 6: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:24.034184 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:24.031988 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.000851 15535 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.036439 15542 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:24.036628 11409 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:24.037056 15478 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.036963 15542 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:24.037572 15478 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:23.996464 15511 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:23.995954 15595 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:23.950139 15531 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.035096 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:24.039513 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 97281.6us
I20260709 13:16:24.040187 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:24.043431 11511 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:24.044066 12034 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:24.046591 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:23.950368 15589 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.016018 15092 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:24.014351 15486 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:24.079758 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 956) took 1870 ms. Trace:
I20260709 13:16:24.032871 15541 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.005319 15572 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.006795 15606 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:24.082510 11409 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:24.109597 12060 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:24.016817 15223 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:16:24.081355 15508 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:24.090353 11368 rpcz_store.cc:276] 0709 13:16:22.209726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.211375 (+  1649us) service_pool.cc:224] Handling call
0709 13:16:22.231947 (+ 20572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:22.308974 (+ 77027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.309006 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.317296 (+  8290us) log.cc:844] Serialized 26 byte log entry
0709 13:16:23.059282 (+741986us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b1800350da0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:23.294299 (+235017us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:23.294345 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.294421 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:24.045947 (+751526us) raft_consensus.cc:1685] finished
0709 13:16:24.045966 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.079675 (+ 33709us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1873,"raft.queue_time_us":5259,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":8670,"spinlock_wait_cycles":48163200,"thread_start_us":234818,"threads_started":1,"wal-append.queue_time_us":1135221}
I20260709 13:16:24.082031 11438 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:16:24.107751 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:24.130012 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:24.130406 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.133026 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:24.133935 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.943475 15577 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.108652 11362 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 6.
I20260709 13:16:24.111131 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:24.014132 15362 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.138676 11370 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:24.139183 15605 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.140116 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.141099 15223 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:24.141281 15362 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.143350 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.144778 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.146042 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:24.146975 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.147280 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:24.116228 11515 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:24.116954 12064 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:24.117251 15616 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.119892 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:24.120844 12066 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:24.036518 15504 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:24.122501 15587 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
W20260709 13:16:24.124532 15599 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:16:24.125710 15509 consensus_meta.cc:287] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.578s	user 0.005s	sys 0.003s
I20260709 13:16:24.126066 15606 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.086009 15232 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:24.108739 15213 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:24.147334 11370 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:24.151473 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:24.151726 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:24.108589 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:24.159436 12062 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:24.113998 15513 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:24.155536 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.114564 15536 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:24.155277 15223 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:24.085609 15361 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:24.128079 15508 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:24.170042 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:24.172099 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.183157 11362 raft_consensus.cc:2376] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10. Re-sending same reply.
I20260709 13:16:24.183317 11440 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:24.172220 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.040676 15604 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.166733 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:24.163491 15598 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:24.179656 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.155472 15092 raft_consensus.cc:2764] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:16:24.181860 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:24.159950 15535 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:16:24.175146 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 66654.8us
W20260709 13:16:24.186272 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.186944 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:24.176549 15609 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:16:24.174489 15504 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:24.198804 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:24.177037 15600 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:16:24.196290 11335 leader_election.cc:400] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6.
I20260709 13:16:24.195173 12034 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:24.176765 15513 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:24.174618 15542 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:24.165498 15602 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:16:24.208914 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:24.185218 15508 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:24.172727 15605 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:16:24.194677 15254 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:24.195983 15535 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:24.198549 15498 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:24.237007 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:24.166961 15509 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.233496 11335 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:24.232324 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 85170.1us
I20260709 13:16:24.208040 15513 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:24.391391 11443 raft_consensus.cc:2368] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 5. Current term is 6.
W20260709 13:16:24.396282 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.396814 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:24.399801 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.168337 15616 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.198208 15544 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 13:16:24.406960 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:24.444092 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44421.7us
I20260709 13:16:24.448604 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:24.450352 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.230433 15613 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:24.445416 15232 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:16:24.452070 15220 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:24.178392 15486 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.234669 15430 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:24.233995 15602 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:24.455611 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:24.456759 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 259352us
I20260709 13:16:24.460024 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:24.234830 15535 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.183987 15511 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:24.225746 15542 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.221s
I20260709 13:16:24.394618 15605 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:24.516672 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 286560us
I20260709 13:16:24.463076 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.515341 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 821) took 2175 ms. Trace:
I20260709 13:16:24.399116 15092 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.541091 11362 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:24.400465 15513 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:24.589967 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:24.591089 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:24.546447 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 83495.1us
I20260709 13:16:24.591841 11362 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:24.592034 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:24.592576 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:24.447207 15223 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:24.537107 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.391288 15254 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.398532 15362 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:16:24.598439 15362 consensus_queue.cc:1059] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:24.539386 11436 rpcz_store.cc:276] 0709 13:16:22.316839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.317696 (+   857us) service_pool.cc:224] Handling call
0709 13:16:22.317998 (+   302us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:22.363826 (+ 45828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.363861 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:22.379829 (+ 15968us) log.cc:844] Serialized 52 byte log entry
0709 13:16:23.562041 (+1182212us) spinlock_profiling.cc:243] Waited 9.3 ms on lock 0x7b18003da200. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:23.759828 (+197787us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:23.759870 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.761193 (+  1323us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:24.457650 (+696457us) raft_consensus.cc:1685] finished
0709 13:16:24.457674 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.492257 (+ 34583us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13323,"raft.queue_time_us":12665,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":13259,"spinlock_wait_cycles":21379840,"thread_start_us":197580,"threads_started":1,"wal-append.queue_time_us":1449292}
I20260709 13:16:24.456722 15405 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:24.600714 15405 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:24.601146 15405 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:24.457172 15442 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.603519 15513 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:24.603791 15442 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:24.593287 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 359592us
I20260709 13:16:24.606325 15442 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:24.563443 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2365) took 1120 ms. Trace:
I20260709 13:16:24.570686 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.481282 15606 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.486934 15613 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:24.445883 15498 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.594877 15223 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.609261 11368 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:24.610144 15498 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.600237 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.608431 15613 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.452008 15614 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.442272 15478 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:24.592885 15220 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:24.597299 15254 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.606942 11367 rpcz_store.cc:276] 0709 13:16:23.442627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.443024 (+   397us) service_pool.cc:224] Handling call
0709 13:16:24.563237 (+1120213us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000a73d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:24.563362 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321499520}
I20260709 13:16:24.599360 15362 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:24.613843 15362 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:24.614393 11367 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:24.614688 15362 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.615926 15362 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:24.615844 15614 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.618158 15614 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.536780 15602 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:24.612275 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.628334 11436 raft_consensus.cc:2410] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:16:24.558264 15232 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:24.621059 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.621685 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:24.594599 11362 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:24.622260 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.622839 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:24.615963 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:24.403141 15627 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.330s
I20260709 13:16:24.635076 11444 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:24.629113 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:24.636126 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 67658.5us
I20260709 13:16:24.635915 11438 raft_consensus.cc:2376] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9. Re-sending same reply.
I20260709 13:16:24.556650 15605 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.487553 15598 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:24.630570 11365 consensus_meta.cc:287] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.615s	user 0.004s	sys 0.005s
I20260709 13:16:24.633663 11445 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:24.634162 11439 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:24.584233 15465 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.000s
I20260709 13:16:24.559013 15531 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:24.633621 11362 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:24.619065 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:24.646957 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 202081us
I20260709 13:16:24.629420 15572 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:24.710083 11511 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:24.710572 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.714583 11365 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:24.709064 11362 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.636802 15544 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:24.636704 15325 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:24.636291 15092 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.905416 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:24.903766 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 290221us
W20260709 13:16:24.815171 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 244559us
I20260709 13:16:24.934681 12062 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:24.820406 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:24.678521 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.935497 12061 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:24.930235 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 393221us
W20260709 13:16:24.769734 15631 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:16:24.723757 12060 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:16:24.935976 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.778368 11511 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
W20260709 13:16:24.937978 11518 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.959s	user 0.007s	sys 0.015s
W20260709 13:16:24.909024 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 419062us
I20260709 13:16:24.952773 11518 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
W20260709 13:16:24.771318 15621 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 13:16:24.933174 11362 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:24.947362 15598 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:24.936810 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.637256 15589 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:24.771970 15628 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 13:16:24.937662 15220 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:24.841151 15624 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 13:16:24.953409 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1748) took 1011 ms. Trace:
I20260709 13:16:25.078868 11518 rpcz_store.cc:276] 0709 13:16:23.942003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.942116 (+   113us) service_pool.cc:224] Handling call
0709 13:16:24.953351 (+1011235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.080204 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:24.647241 15614 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:24.933269 11445 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:24.841866 15500 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:16:24.841678 15625 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
W20260709 13:16:25.012750 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.952533 15616 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:24.935878 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:24.956192 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:24.738444 15232 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:24.909004 15427 consensus_meta.cc:287] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.943s	user 0.009s	sys 0.000s
I20260709 13:16:25.115063 11511 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:24.799042 15607 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:24.769836 15634 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:24.963534 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:25.116001 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.116575 11511 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:24.764730 15637 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.313s
I20260709 13:16:24.947147 15254 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:16:24.945824 15531 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:25.082717 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 374425us
I20260709 13:16:24.730491 15465 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:25.094249 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.098064 12061 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:25.132236 11362 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:16:25.111673 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.111673 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.115136 15232 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.116990 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1756) took 1157 ms. Trace:
I20260709 13:16:24.797019 15542 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:25.117416 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 181607us
I20260709 13:16:25.135004 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:24.964234 15589 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.804925 15640 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.148057 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:25.149757 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:24.945923 15325 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.151309 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:25.092105 15500 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:25.152293 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2264) took 1029 ms. Trace:
I20260709 13:16:25.168424 11518 rpcz_store.cc:276] 0709 13:16:24.122830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.111359 (+988529us) service_pool.cc:224] Handling call
0709 13:16:25.152221 (+ 40862us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:25.133725 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 170256us
I20260709 13:16:24.960850 15638 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.136596 15584 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.136857 15630 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:25.138201 15641 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:16:25.138792 15649 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 13:16:25.180361 15407 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 1.190s	user 0.010s	sys 0.001s
I20260709 13:16:25.139423 15092 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.140090 15542 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.211743 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:25.146456 15531 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:25.140020 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:25.217602 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.169713 11366 raft_consensus.cc:2368] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10.
I20260709 13:16:25.172636 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.222590 12060 raft_consensus.cc:2410] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:16:25.223593 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.224778 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.225871 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.122874 15614 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:25.226936 15614 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.227510 15614 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.229653 15614 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.230175 15614 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.132726 11362 pending_rounds.cc:85] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 7
I20260709 13:16:25.232074 11362 pending_rounds.cc:107] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:16:25.232357 11362 raft_consensus.cc:2889] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:25.233080 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2387) took 1084 ms. Trace:
I20260709 13:16:25.233372 11362 rpcz_store.cc:276] 0709 13:16:24.148820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.594224 (+445404us) service_pool.cc:224] Handling call
0709 13:16:24.594497 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.232802 (+638305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.233022 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.234615 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:25.177086 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1746) took 1235 ms. Trace:
I20260709 13:16:25.235993 11516 rpcz_store.cc:276] 0709 13:16:23.941396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.950139 (+  8743us) service_pool.cc:224] Handling call
0709 13:16:23.978732 (+ 28593us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:24.005787 (+ 27055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.005815 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:24.006285 (+   470us) log.cc:844] Serialized 52 byte log entry
0709 13:16:24.133993 (+127708us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:24.134053 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.134161 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.176821 (+1042660us) raft_consensus.cc:1685] finished
0709 13:16:25.176843 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.177015 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5547,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":12360,"thread_start_us":127172,"threads_started":1,"wal-append.queue_time_us":196029}
I20260709 13:16:25.237835 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.238567 11516 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:25.238943 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 921) took 1118 ms. Trace:
I20260709 13:16:25.179832 11369 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:24.797223 15602 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.240401 15602 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.152052 15627 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:25.162338 15607 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:25.244017 15607 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.244448 15607 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:25.172937 15535 consensus_meta.cc:287] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
I20260709 13:16:25.247678 15535 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.249897 15535 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.189273 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:25.250927 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 61745.1us
I20260709 13:16:25.251603 11436 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:25.253280 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:25.154672 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.254606 12061 raft_consensus.cc:2393] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:25.230603 15614 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.257802 15614 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.196054 15361 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:25.115232 15427 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.198117 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2252) took 3540 ms. Trace:
I20260709 13:16:25.196436 15643 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:25.198742 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1848) took 1448 ms. Trace:
I20260709 13:16:25.205456 15649 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:25.206252 15448 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.206590 15325 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:25.206740 15659 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:25.153573 15589 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.207554 15642 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.210484 15407 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:25.210937 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 910) took 1345 ms. Trace:
I20260709 13:16:25.214476 11518 raft_consensus.cc:2368] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:25.134433 11511 rpcz_store.cc:276] 0709 13:16:23.959346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.115741 (+1156395us) service_pool.cc:224] Handling call
0709 13:16:25.116939 (+  1198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.207468 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.258271 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:25.191318 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 235002us
I20260709 13:16:25.258980 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1193) took 4116 ms. Trace:
I20260709 13:16:25.196271 15635 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:25.235206 11362 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.203629 15605 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.203986 15584 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.203367 15638 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.166734 15640 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.265620 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:25.427418 11367 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:16:25.426841 11370 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2376) took 1535 ms (client timeout 1940 ms). Trace:
W20260709 13:16:25.429227 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 277972us
W20260709 13:16:25.321653 12035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.052s	sys 0.026s
I20260709 13:16:25.428145 11484 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:25.432235 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.239216 11516 rpcz_store.cc:276] 0709 13:16:24.120150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.237552 (+1117402us) service_pool.cc:224] Handling call
0709 13:16:25.238881 (+  1329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.428877 11510 rpcz_store.cc:276] 0709 13:16:21.658096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.711476 (+1053380us) service_pool.cc:224] Handling call
0709 13:16:22.745043 (+ 33567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.715631 (+970588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.197894 (+1482263us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54000ea954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:25.198060 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2207844224}
W20260709 13:16:25.387159 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.449795 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:25.450073 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 184557us
W20260709 13:16:25.447113 11370 rpcz_store.cc:269] 0709 13:16:23.891907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.953833 (+ 61926us) service_pool.cc:224] Handling call
0709 13:16:25.426642 (+1472809us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54000c7454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:25.426774 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327333632}
W20260709 13:16:25.425810 11410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.010s	sys 0.006s
W20260709 13:16:25.451498 11410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.010s	sys 0.007s
W20260709 13:16:25.448925 12035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.057s	sys 0.032s
I20260709 13:16:25.428808 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 917) took 1375 ms. Trace:
W20260709 13:16:25.452163 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.452279 11518 rpcz_store.cc:276] 0709 13:16:24.053246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.211470 (+1158224us) service_pool.cc:224] Handling call
0709 13:16:25.428737 (+217267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.429154 11514 rpcz_store.cc:276] 0709 13:16:23.865671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.866244 (+   573us) service_pool.cc:224] Handling call
0709 13:16:23.903238 (+ 36994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:24.126234 (+222996us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:24.126264 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:24.126805 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:16:24.181772 (+ 54967us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:24.181816 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.210559 (+1028743us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54002c06d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:25.210607 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.210655 (+    48us) raft_consensus.cc:1685] finished
0709 13:16:25.210668 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.210881 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21503,"raft.queue_time_us":22964,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":4505,"spinlock_wait_cycles":462351232,"thread_start_us":54577,"threads_started":1,"wal-append.queue_time_us":316031}
I20260709 13:16:25.299881 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:25.457679 12068 rpcz_store.cc:276] 0709 13:16:21.103797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.108839 (+  5042us) service_pool.cc:224] Handling call
0709 13:16:21.914502 (+805663us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:16:21.914695 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.833771 (+1919076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.215119 (+1381348us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54001faed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:25.215313 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4340327552}
I20260709 13:16:25.459923 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.460597 11370 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:25.461202 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.458242 11445 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:25.388942 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.461881 11511 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:25.446972 11363 rpcz_store.cc:276] 0709 13:16:23.750807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.750925 (+   118us) service_pool.cc:224] Handling call
0709 13:16:23.751213 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:23.949510 (+198297us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:23.949540 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:23.950017 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:16:24.169140 (+219123us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:24.169184 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:24.169258 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.198519 (+1029261us) raft_consensus.cc:1685] finished
0709 13:16:25.198539 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.198685 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93112,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":34585,"thread_start_us":369527,"threads_started":2,"wal-append.queue_time_us":229976}
I20260709 13:16:25.214943 15511 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.467032 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.466818 15511 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.467734 11363 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:25.468611 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.469101 11363 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:25.470815 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:25.471355 11363 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.478657 11363 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:25.479699 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:25.485211 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:25.480931 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2259) took 1359 ms. Trace:
W20260709 13:16:25.293627 15595 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 1.213s	user 0.008s	sys 0.000s
I20260709 13:16:25.483254 11439 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:25.426812 15634 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:25.487027 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:25.456328 15508 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.225s	user 0.005s	sys 0.012s
I20260709 13:16:25.486994 11511 rpcz_store.cc:276] 0709 13:16:24.121527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.456520 (+1334993us) service_pool.cc:224] Handling call
0709 13:16:25.480864 (+ 24344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.455516 15616 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.456424 15504 consensus_meta.cc:287] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 1.248s	user 0.017s	sys 0.003s
I20260709 13:16:25.264509 15509 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.516158 12063 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:16:25.503171 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 249954us
I20260709 13:16:25.504328 12068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:25.517115 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:25.323019 15213 consensus_meta.cc:287] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 1.139s	user 0.005s	sys 0.005s
I20260709 13:16:25.455693 15605 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:25.495837 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:25.490854 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.524719 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.525552 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2270) took 1375 ms. Trace:
W20260709 13:16:25.491341 15541 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.379s	user 0.008s	sys 0.003s
I20260709 13:16:25.464614 15662 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:25.492620 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.509260 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.511222 11445 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
W20260709 13:16:25.501456 12062 consensus_meta.cc:287] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.003s
I20260709 13:16:25.464897 15602 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:25.460150 15627 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:25.513121 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.517187 12068 raft_consensus.cc:606] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:25.531584 11483 leader_election.cc:400] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9.
I20260709 13:16:25.519333 11362 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:25.299132 15535 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.539922 11442 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:25.525879 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:25.527681 11517 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:25.541273 11363 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.527204 12062 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:25.526816 11511 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:25.543222 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 922) took 1422 ms. Trace:
I20260709 13:16:25.525867 11516 rpcz_store.cc:276] 0709 13:16:24.149720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.500786 (+1351066us) service_pool.cc:224] Handling call
0709 13:16:25.525497 (+ 24711us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.543629 11511 rpcz_store.cc:276] 0709 13:16:24.120422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.504942 (+1384520us) service_pool.cc:224] Handling call
0709 13:16:25.543159 (+ 38217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.508905 15478 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:25.535248 11362 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:16:25.515805 15506 consensus_meta.cc:287] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 1.926s	user 0.012s	sys 0.001s
I20260709 13:16:25.552547 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.537004 15427 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.538225 15658 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:25.427073 15607 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.490514 15654 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.260784 15220 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.558586 12068 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:25.533579 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:25.547436 12062 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 6. Current term is 7. Ops: []
W20260709 13:16:25.548838 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 86188.3us
I20260709 13:16:25.553414 11511 raft_consensus.cc:2410] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:25.554975 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.460656 15254 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:16:25.530256 15213 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:25.555555 11363 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:25.592620 11362 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:25.459033 15642 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.520279 15465 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.531584 15662 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.495452 15646 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:25.598388 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:25.557415 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.462455 15643 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.594275 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1759) took 1568 ms. Trace:
I20260709 13:16:25.594520 11370 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:16:25.559000 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.515787 15508 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.596333 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:25.539353 15602 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.546864 15504 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:16:25.521955 15616 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.600337 11516 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:25.602263 11511 rpcz_store.cc:276] 0709 13:16:24.025719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.548785 (+1523066us) service_pool.cc:224] Handling call
0709 13:16:25.594218 (+ 45433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.555496 15658 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:25.602752 12060 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:25.606288 15665 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:25.610756 15642 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.611258 15361 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.626354 15361 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.612273 15589 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:25.599805 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 66299.8us
W20260709 13:16:25.627983 15655 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
I20260709 13:16:25.628283 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:25.630277 15640 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.630633 15361 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.631824 15361 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.632555 15361 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.634858 15361 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.613756 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.502733 15595 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.636965 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.637709 15595 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:25.624907 11370 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:25.635253 15531 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:25.635517 15640 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.638119 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.640015 11445 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:25.639973 15640 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.640938 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.641955 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.595336 15663 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:25.642962 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.598445 15646 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.613489 15643 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:25.645735 11445 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:25.642771 15663 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.649031 11370 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:25.598449 15662 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.655441 12067 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:25.662783 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:25.614238 15542 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.548949 15535 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.642052 15640 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.624794 15654 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.616225 15602 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.665146 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.684150 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.617897 15581 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.730854 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:25.752780 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 124577us
I20260709 13:16:25.753748 11370 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
W20260709 13:16:25.642266 15405 consensus_meta.cc:287] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 1.041s	user 0.005s	sys 0.005s
I20260709 13:16:25.642138 15638 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:25.618728 15465 raft_consensus.cc:2764] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:16:25.758687 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:25.623353 15634 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:25.624541 15658 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.761731 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.663429 15646 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:25.684856 15595 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:25.752146 15427 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.731452 15325 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.556483 15506 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.730521 15512 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.767026 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:25.767356 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:25.761694 15584 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.809239 11362 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:25.768111 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.773984 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 911) took 1908 ms. Trace:
I20260709 13:16:25.701601 15478 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:25.810242 11515 rpcz_store.cc:276] 0709 13:16:23.865935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.892089 (+ 26154us) service_pool.cc:224] Handling call
0709 13:16:23.926024 (+ 33935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.147968 (+221944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.773720 (+1625752us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400197dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:25.773920 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436495104}
I20260709 13:16:25.811434 11511 raft_consensus.cc:2410] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:25.805092 15448 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.813843 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 915) took 1819 ms. Trace:
I20260709 13:16:25.814795 11511 rpcz_store.cc:276] 0709 13:16:23.994289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.759034 (+1764745us) service_pool.cc:224] Handling call
0709 13:16:25.813779 (+ 54745us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:25.816255 15448 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.814013 15512 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.818506 15448 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.819170 15512 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.816939 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.812362 15427 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.820463 11515 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:25.821087 15616 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:25.821404 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:25.821955 11515 raft_consensus.cc:2376] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14. Re-sending same reply.
I20260709 13:16:25.814599 15213 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:25.825430 15213 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.825795 15213 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:25.828016 15663 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:25.830368 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 63401.9us
I20260709 13:16:25.831283 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:25.832947 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.833564 11515 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:25.766674 15542 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:25.730525 15643 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:16:25.839913 11370 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:25.837594 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:57392 (request call id 11) took 1721 ms. Trace:
I20260709 13:16:25.840418 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:25.839880 11513 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:25.840633 11370 raft_consensus.cc:606] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:25.841959 11370 raft_consensus.cc:614] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:25.840687 12069 rpcz_store.cc:276] 0709 13:16:24.108113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.108233 (+   120us) service_pool.cc:224] Handling call
0709 13:16:25.828809 (+1720576us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54007783d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca04735 00007f35bc9fc9fa 00007f35c19c44c9 00007f35baf40c54 00007f35baf40be2 00007f35baf40b11 00007f35baf40a8c 00007f35baf3fd42 00007f35bad3948c 00007f35bad388c6
0709 13:16:25.829039 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3396547456}
I20260709 13:16:25.806643 15634 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:25.763912 15405 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.847719 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:25.846864 15362 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 1.230s	user 0.009s	sys 0.002s
I20260709 13:16:25.849642 15589 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:25.837134 15357 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.850596 15357 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.852597 15357 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:25.853483 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.837433 15087 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:25.812234 15325 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.856709 15508 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:25.857714 15325 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.858328 15535 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:25.858606 15592 log.cc:927] Time spent T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 2.064s	user 0.007s	sys 0.003s
I20260709 13:16:25.862432 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2529) took 2493 ms. Trace:
I20260709 13:16:25.862782 11437 rpcz_store.cc:276] 0709 13:16:23.368842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.370120 (+  1278us) service_pool.cc:224] Handling call
0709 13:16:23.370393 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:23.375267 (+  4874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.375300 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:23.400708 (+ 25408us) log.cc:844] Serialized 26 byte log entry
0709 13:16:23.542171 (+141463us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:23.542219 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.542293 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.858316 (+2316023us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b240040c7a8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b97979c4 00007f35b9795acb 00007f35b9950dc9 00007f35b9950a55 00007f35b995009d 00007f35bc58210e 00007f35bc59988c 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221
0709 13:16:25.862160 (+  3844us) raft_consensus.cc:1685] finished
0709 13:16:25.862179 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.862376 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18229,"raft.queue_time_us":1276,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":4333,"spinlock_wait_cycles":47283072,"thread_start_us":99688,"threads_started":1,"wal-append.queue_time_us":371913}
I20260709 13:16:25.858774 15649 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:25.864880 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.865926 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:25.867017 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:25.868341 11437 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:25.841291 15595 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.872594 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.871806 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.875010 11362 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:25.876091 11515 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:25.876924 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:25.879056 11516 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:25.861266 15365 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:25.884377 15513 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:25.886456 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:25.888437 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.888599 15658 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.889604 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:25.890379 11444 raft_consensus.cc:2410] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:25.897347 11437 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:25.768498 15581 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.901880 11362 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:16:25.868431 15542 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:25.908996 15531 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:25.909814 11364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 976) took 2447 ms (client timeout 2615 ms). Trace:
I20260709 13:16:25.903898 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1857) took 1282 ms. Trace:
I20260709 13:16:25.910727 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:25.910612 11363 rpcz_store.cc:276] 0709 13:16:24.621080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.592576 (+971496us) service_pool.cc:224] Handling call
0709 13:16:25.592878 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.593236 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.593262 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.599760 (+  6498us) log.cc:844] Serialized 26 byte log entry
0709 13:16:25.833145 (+233385us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:25.833189 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.833261 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.903707 (+ 70446us) raft_consensus.cc:1685] finished
0709 13:16:25.903727 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.903844 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":655,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10973,"thread_start_us":223676,"threads_started":1,"wal-append.queue_time_us":286643}
W20260709 13:16:25.910177 11364 rpcz_store.cc:269] 0709 13:16:23.462223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.468475 (+  6252us) service_pool.cc:224] Handling call
0709 13:16:25.909624 (+2441149us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b540035b454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:25.909738 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477904768}
I20260709 13:16:25.909610 15542 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.915174 15604 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:25.920792 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 43921us
I20260709 13:16:25.949714 11445 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.922535 15571 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:25.956224 15571 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:25.957132 15571 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:25.957310 15634 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:25.957525 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:25.959079 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.959277 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.961122 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.961894 12064 raft_consensus.cc:2410] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:25.962652 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.964349 12033 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:25.964649 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.967378 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:25.967572 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.963299 11515 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:25.967963 12065 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:25.977202 12065 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:25.978511 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:25.912552 15607 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:25.985459 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.986297 11362 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:16:25.984097 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.987622 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.990092 11445 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:25.958166 15571 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:25.994167 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.994949 12064 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:25.959627 15513 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.961630 15593 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:16:26.018428 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:25.961710 15504 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:25.976337 15499 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.020853 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:25.983916 15388 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.984159 15658 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:26.024587 11364 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:25.983744 15517 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.991722 15616 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:26.031343 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:25.900863 15478 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:26.036782 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 93317.3us
I20260709 13:16:26.037035 12064 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:26.037585 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.036710 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.039014 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.044047 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:25.992692 15637 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:26.045939 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 66324.8us
I20260709 13:16:25.991482 15649 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:16:26.047163 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.050132 11364 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:25.991633 15656 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:16:26.052320 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:26.052755 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.052925 11362 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.023885 15513 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.042260 15616 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:26.048518 15637 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.031360 15604 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:26.067725 12064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:26.070767 12064 raft_consensus.cc:606] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:26.071223 12064 raft_consensus.cc:614] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:26.029642 15504 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.036592 15388 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.075369 15658 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:26.078472 15427 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.083420 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.084219 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.084260 12061 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:26.085541 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.084482 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1053) took 1004 ms. Trace:
I20260709 13:16:26.086939 12063 rpcz_store.cc:276] 0709 13:16:25.079553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.529845 (+450292us) service_pool.cc:224] Handling call
0709 13:16:25.554063 (+ 24218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.615145 (+ 61082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.615173 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:25.615211 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:25.615244 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.084234 (+468990us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b54004a65d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:26.084284 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.084422 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140028160}
W20260709 13:16:26.090003 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.059916 15654 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:26.091531 11442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2372) took 1937 ms (client timeout 1605 ms). Trace:
I20260709 13:16:26.031494 15499 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.081295 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.082135 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.082211 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:26.091959 11442 rpcz_store.cc:269] 0709 13:16:24.154139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.155276 (+  1137us) service_pool.cc:224] Handling call
0709 13:16:26.091306 (+1936030us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002094d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.091456 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3139280128}
I20260709 13:16:26.102324 12065 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:26.103044 11442 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:26.104697 11442 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:26.106105 11442 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:26.079604 15232 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.043515 15517 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:26.106518 11442 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:26.060802 15405 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:26.066437 15616 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:26.120061 11362 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:26.072259 15637 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:25.991183 15676 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 13:16:26.121496 11442 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:26.128218 11442 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:26.135787 15506 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.124297 15595 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:26.155582 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.159549 11510 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:26.171435 15581 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.172103 15388 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.174448 15602 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.176911 15654 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.177515 15654 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.152930 15213 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.181699 15213 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:26.188463 15517 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:26.192188 15513 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.194921 15607 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.195701 12063 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:26.195564 15607 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.197806 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:26.188760 11365 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 968) took 2734 ms (client timeout 1589 ms). Trace:
W20260709 13:16:26.199110 11365 rpcz_store.cc:269] 0709 13:16:23.454472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.454570 (+    98us) service_pool.cc:224] Handling call
0709 13:16:26.188564 (+2733994us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b5400249354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.188681 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275504896}
W20260709 13:16:26.203642 11484 leader_election.cc:336] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.631s (SENT)
I20260709 13:16:26.188467 15504 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.205683 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.208148 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:26.209473 15499 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:26.209697 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2370) took 2058 ms (client timeout 1704 ms). Trace:
I20260709 13:16:26.210496 15582 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:26.198554 12064 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:16:26.210055 11443 rpcz_store.cc:269] 0709 13:16:24.151015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.151138 (+   123us) service_pool.cc:224] Handling call
0709 13:16:26.209491 (+2058353us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54003147d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.209629 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144286464}
I20260709 13:16:26.208742 15513 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:26.212352 15513 consensus_queue.cc:1243] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:26.198596 15607 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.176901 15602 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.210978 11437 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:16:26.208485 11519 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:26.215806 15507 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.216277 15507 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:26.204432 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:26.198719 15531 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:26.210140 15581 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:26.218947 15582 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:26.221349 15581 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:26.222332 15581 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.224079 15662 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:26.217362 15646 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.225680 15608 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:26.226121 15608 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:26.227041 15646 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:26.228330 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:26.232620 11510 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:26.226884 15608 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.253106 15608 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:26.257332 15637 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.231693 15607 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:26.232820 15465 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:26.257457 15362 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:16:26.227041 15662 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:26.276625 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.250401 15616 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.281760 15616 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:26.260150 15637 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:26.289305 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 163001us
W20260709 13:16:26.292203 11483 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 2.094s (SENT) [suppressed 40 similar messages]
I20260709 13:16:26.332168 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:26.333108 12065 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:26.333833 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:26.337966 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.339318 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.341405 12067 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:16:26.359238 12067 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:26.390365 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.467083 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:26.467751 12061 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:26.485670 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:26.508195 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.537258 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:26.538100 11484 leader_election.cc:336] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Timed out: RequestConsensusVote RPC to 127.10.4.66:44257 timed out after 1.968s (SENT)
I20260709 13:16:26.538520 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:26.538580 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.539013 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.635195 12065 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:26.560103 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.552371 11511 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:26.574196 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1849) took 2822 ms. Trace:
W20260709 13:16:26.540005 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.569715 12061 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:26.640106 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:26.639012 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.639652 11369 rpcz_store.cc:276] 0709 13:16:23.752237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.948660 (+196423us) service_pool.cc:224] Handling call
0709 13:16:23.977560 (+ 28900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.239669 (+1262109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.573940 (+1334271us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000dbe54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:26.574112 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2755596032}
I20260709 13:16:26.640863 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.642185 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.642387 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.642211 12067 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:26.642989 11364 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:26.646318 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.647043 11365 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:26.648005 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:26.648496 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1847) took 2898 ms. Trace:
I20260709 13:16:26.649024 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.649628 12065 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:26.648860 11361 rpcz_store.cc:276] 0709 13:16:23.750493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.750627 (+   134us) service_pool.cc:224] Handling call
0709 13:16:25.910727 (+2160100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.911062 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.911089 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.914611 (+  3522us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.040958 (+126347us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:26.041004 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.041077 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.153241 (+112164us) raft_consensus.cc:1685] finished
0709 13:16:26.153261 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.648244 (+494983us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b540035b450. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.648438 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":464,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":1337,"spinlock_wait_cycles":4967103360,"thread_start_us":126007,"threads_started":1,"wal-append.queue_time_us":168617}
W20260709 13:16:26.650837 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 112769us
I20260709 13:16:26.652468 11408 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:26.652738 11514 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:26.655088 12064 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:26.654780 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:26.654248 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.656208 11443 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:26.657011 11443 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:26.660199 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.660696 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:26.661233 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.660904 12061 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:26.538751 15648 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.610543 15465 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.664525 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1753) took 2711 ms. Trace:
I20260709 13:16:26.664907 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:26.669049 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.679971 11514 rpcz_store.cc:276] 0709 13:16:23.943297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.652145 (+2708848us) service_pool.cc:224] Handling call
0709 13:16:26.652443 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.654620 (+  2177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.654767 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:26.685395 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.686810 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.688866 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.659838 15582 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:26.689816 11514 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:26.664759 15656 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:26.686764 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:26.675418 15362 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:26.692301 11365 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:26.687330 15232 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:16:26.692142 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 913) took 2748 ms. Trace:
I20260709 13:16:26.700361 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.681541 15648 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.684729 15465 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.684003 15607 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:26.709769 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.711913 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:26.720453 11519 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1726) took 5226 ms (client timeout 1532 ms). Trace:
W20260709 13:16:26.744251 11519 rpcz_store.cc:269] 0709 13:16:21.493703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.510111 (+ 16408us) service_pool.cc:224] Handling call
0709 13:16:26.720203 (+5210092us) spinlock_profiling.cc:243] Waited 4.52 s on lock 0x7b54003d8e54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.720347 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10393056640}
W20260709 13:16:26.739434 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52722.7us
I20260709 13:16:26.721871 11512 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:26.725358 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1765) took 1234 ms. Trace:
I20260709 13:16:26.746076 11511 rpcz_store.cc:276] 0709 13:16:25.491145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.849155 (+358010us) service_pool.cc:224] Handling call
0709 13:16:26.725199 (+876044us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b540031e554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.725306 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605811456}
I20260709 13:16:26.727435 15654 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:26.729017 12062 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:16:26.729631 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.739015 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.748607 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:26.748910 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.749315 11443 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:26.750078 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.739130 11514 rpcz_store.cc:276] 0709 13:16:23.943989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.684187 (+2740198us) service_pool.cc:224] Handling call
0709 13:16:26.689662 (+  5475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.691822 (+  2160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.692093 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:26.723747 15486 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:26.743539 15582 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:26.751345 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 916) took 2734 ms. Trace:
I20260709 13:16:26.751706 11519 rpcz_store.cc:276] 0709 13:16:24.016633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.745224 (+2728591us) service_pool.cc:224] Handling call
0709 13:16:26.745513 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.751090 (+  5577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.751116 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:26.751159 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:26.751190 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.751229 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.751297 (+    68us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:26.752203 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.755549 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 919) took 2636 ms. Trace:
I20260709 13:16:26.755844 11511 rpcz_store.cc:276] 0709 13:16:24.119497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.746802 (+2627305us) service_pool.cc:224] Handling call
0709 13:16:26.755151 (+  8349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.755277 (+   126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.755291 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:26.755316 (+    25us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:26.755339 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.755384 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.755502 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:26.746881 11409 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:26.743733 15362 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:26.757761 15427 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:26.758289 15427 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:26.758162 15362 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.758181 11511 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:26.759832 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 924) took 2638 ms. Trace:
I20260709 13:16:26.760115 11511 rpcz_store.cc:276] 0709 13:16:24.120957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.756497 (+2635540us) service_pool.cc:224] Handling call
0709 13:16:26.757994 (+  1497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.759624 (+  1630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.759788 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:26.759191 15427 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:26.762326 15223 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.764572 15465 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.765268 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:26.766587 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.766234 15500 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:26.766959 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.767185 15500 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:26.769480 12061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:26.770006 12061 raft_consensus.cc:606] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:26.770766 15511 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.771204 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:26.772334 15648 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:26.747094 15607 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.770952 11408 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:26.774673 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.776579 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.776588 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:26.752887 15582 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:26.781289 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:26.786728 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2381) took 1160 ms. Trace:
I20260709 13:16:26.787230 11442 rpcz_store.cc:276] 0709 13:16:25.626455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.217532 (+591077us) service_pool.cc:224] Handling call
0709 13:16:26.217949 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.218284 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.218313 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.228293 (+  9980us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.455884 (+227591us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:26.536628 (+ 80744us) log.cc:844] Serialized 14 byte log entry
0709 13:16:26.536843 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.536926 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.786527 (+249601us) raft_consensus.cc:1685] finished
0709 13:16:26.786545 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.786672 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":83103,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":3149,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":5682,"thread_start_us":176445,"threads_started":1,"wal-append.queue_time_us":532366}
I20260709 13:16:26.789139 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.789723 11442 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:26.790073 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.790906 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:26.791460 12062 raft_consensus.cc:2376] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10. Re-sending same reply.
I20260709 13:16:26.791477 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:26.789145 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.793301 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.776906 15607 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:26.795367 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:26.779387 15513 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:26.797076 15513 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:26.797466 15513 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:26.797979 15643 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:26.800240 15405 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:26.800598 15405 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:26.801398 15405 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.802301 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.802434 15405 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:26.796284 11256 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.667s	user 0.015s	sys 0.055s
I20260709 13:16:26.796489 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:26.807994 11516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2261) took 2686 ms (client timeout 2819 ms). Trace:
I20260709 13:16:26.808192 11514 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:16:26.808446 11516 rpcz_store.cc:269] 0709 13:16:24.122048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.616527 (+1494479us) service_pool.cc:224] Handling call
0709 13:16:26.807826 (+1191299us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002a80d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.807936 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256531328}
I20260709 13:16:26.784117 15582 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:26.811728 11442 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:26.813133 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.794720 11443 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:26.833835 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:26.834657 11443 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:26.836555 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.837836 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:26.839129 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.841688 11443 raft_consensus.cc:2410] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:16:26.811792 15536 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:26.858639 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.863195 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.864063 11369 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:26.863052 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:26.869182 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:26.870085 11443 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:26.849331 11442 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:26.873418 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:26.868667 15536 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:26.871623 15643 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:26.885355 11438 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:26.875227 11443 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:26.876154 15615 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:26.886274 15536 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:26.888304 12034 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:26.897042 15698 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:26.905580 15698 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:26.906591 15698 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:26.909412 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2545) took 1425 ms. Trace:
I20260709 13:16:26.909873 11445 rpcz_store.cc:276] 0709 13:16:25.483935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.730553 (+246618us) service_pool.cc:224] Handling call
0709 13:16:26.909201 (+1178648us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400438ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:26.909344 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443635584}
I20260709 13:16:26.916523 11445 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:26.918085 11445 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:26.942569 11442 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:26.944236 11442 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:26.947062 11443 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:26.926209 15643 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:16:26.963012 11369 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:26.964267 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:26.979203 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:26.979876 11369 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:26.954062 15654 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:16:26.991627 11369 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:26.963325 15465 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.995069 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:26.983521 15611 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:27.005918 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:27.006573 11369 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:27.010389 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2278) took 1554 ms. Trace:
I20260709 13:16:27.010823 11510 rpcz_store.cc:276] 0709 13:16:25.456118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.886148 (+430030us) service_pool.cc:224] Handling call
0709 13:16:27.010223 (+1124075us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540014d2d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.010323 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626516352}
I20260709 13:16:27.011682 15652 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:27.017843 11510 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:27.019426 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2263) took 2896 ms. Trace:
I20260709 13:16:27.020301 11510 rpcz_store.cc:276] 0709 13:16:24.122583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.017349 (+2894766us) service_pool.cc:224] Handling call
0709 13:16:27.017666 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.019168 (+  1502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.019349 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.023478 11369 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:27.023806 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 834) took 2442 ms. Trace:
I20260709 13:16:27.024366 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 925) took 2903 ms. Trace:
I20260709 13:16:27.024528 12033 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:27.024183 11436 rpcz_store.cc:276] 0709 13:16:24.581298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.650501 (+ 69203us) service_pool.cc:224] Handling call
0709 13:16:24.715128 (+ 64627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.252031 (+536903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.023607 (+1771576us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54003e5dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:27.023754 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233339264}
I20260709 13:16:27.012643 15611 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:27.024923 11511 rpcz_store.cc:276] 0709 13:16:24.121232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.760733 (+2639501us) service_pool.cc:224] Handling call
0709 13:16:26.761008 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.761181 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:26.954584 (+193403us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.023962 (+ 69378us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.024010 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:27.024086 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.024224 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.024321 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":259363,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":29,"raft.queue_time_us":1310,"thread_start_us":261552,"threads_started":2}
I20260709 13:16:27.046332 11515 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:16:27.045791 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 843) took 1722 ms. Trace:
I20260709 13:16:27.050678 11437 rpcz_store.cc:276] 0709 13:16:25.323471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.960284 (+636813us) service_pool.cc:224] Handling call
0709 13:16:26.210682 (+250398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.215296 (+  4614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.045600 (+830304us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54003147d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.045731 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520463104}
W20260709 13:16:27.022872 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b1906ab37ba541f493936685a808c8a9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:27.051456 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:27.033653 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.047215 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:26.927325 15615 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:16:27.017138 15652 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:27.056517 11369 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:27.083949 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:27.084650 11369 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:27.085372 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:27.086000 11369 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:27.004698 15465 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:27.057101 11517 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 909) took 3180 ms (client timeout 2855 ms). Trace:
I20260709 13:16:27.059427 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:27.051518 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:27.091974 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:27.056480 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.056937 15232 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:16:27.021245 15503 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:27.044661 15643 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:27.057165 15665 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.079844 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.048045 15629 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:16:27.081486 11510 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:16:27.080797 11256 maintenance_manager.cc:643] P ab9dad95b38845f197fb7c3c12a492c4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.400s	user 0.266s	sys 0.161s Metrics: {"bytes_written":30370,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":519063,"dirs.run_cpu_time_us":525,"dirs.run_wall_time_us":101787,"drs_written":1,"lbm_read_time_us":382,"lbm_reads_lt_1ms":4,"lbm_write_time_us":130297,"lbm_writes_1-10_ms":4,"lbm_writes_10-100_ms":4,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":477505,"threads_started":2,"update_count":31,"wal-append.queue_time_us":644030}
I20260709 13:16:27.082598 15511 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:27.088016 11511 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:27.089847 11517 rpcz_store.cc:269] 0709 13:16:23.865401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.866169 (+   768us) service_pool.cc:224] Handling call
0709 13:16:27.045967 (+3179798us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54002c1d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.046066 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3548414848}
I20260709 13:16:27.109378 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2268) took 2960 ms. Trace:
I20260709 13:16:27.109759 11517 rpcz_store.cc:276] 0709 13:16:24.148623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.108578 (+2959955us) service_pool.cc:224] Handling call
0709 13:16:27.108881 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.109050 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.109093 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.109132 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:27.109170 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.109217 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.109314 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.091187 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:27.103222 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.104897 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2266) took 2981 ms. Trace:
I20260709 13:16:27.111364 11510 rpcz_store.cc:276] 0709 13:16:24.123305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.023265 (+2899960us) service_pool.cc:224] Handling call
0709 13:16:27.081107 (+ 57842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.104634 (+ 23527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.104839 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.079030 15513 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:27.107656 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:27.112458 11333 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.569s (SENT)
I20260709 13:16:27.113852 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2267) took 2965 ms. Trace:
W20260709 13:16:27.115427 11333 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.605s (SENT)
I20260709 13:16:27.045801 15653 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:16:27.115845 11333 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.114277 11511 rpcz_store.cc:276] 0709 13:16:24.148136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.049989 (+2901853us) service_pool.cc:224] Handling call
0709 13:16:27.087621 (+ 37632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.113623 (+ 26002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.113802 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.108538 15593 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:16:27.101933 15581 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:27.102144 15246 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:27.104368 15500 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:27.137235 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2269) took 2988 ms. Trace:
I20260709 13:16:27.135895 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2271) took 2681 ms. Trace:
I20260709 13:16:27.113155 15511 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.140363 11511 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:16:27.148061 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1760) took 1698 ms. Trace:
I20260709 13:16:27.148414 11511 rpcz_store.cc:276] 0709 13:16:25.450088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.136995 (+1686907us) service_pool.cc:224] Handling call
0709 13:16:27.140129 (+  3134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.147753 (+  7624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.148001 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.143599 11510 rpcz_store.cc:276] 0709 13:16:24.454549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.112124 (+2657575us) service_pool.cc:224] Handling call
0709 13:16:27.135237 (+ 23113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.135513 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.135540 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.135588 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:27.135629 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.135688 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.135822 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.141947 11517 rpcz_store.cc:276] 0709 13:16:24.149154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.110610 (+2961456us) service_pool.cc:224] Handling call
0709 13:16:27.136776 (+ 26166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.136946 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.136973 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.137011 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:27.137045 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.137091 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.137186 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.115767 15643 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:27.119252 15605 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:27.152122 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 923) took 3031 ms. Trace:
I20260709 13:16:27.108346 15619 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:16:27.106825 15654 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:16:27.133287 15581 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:27.162050 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:27.181454 15715 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:16:27.179713 11438 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:27.138582 15362 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:27.106309 15615 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:16:27.181177 11510 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:16:27.204272 11510 pending_rounds.cc:85] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 4
I20260709 13:16:27.204526 11510 pending_rounds.cc:107] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:16:27.204738 11510 raft_consensus.cc:2889] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:27.113884 15665 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.141811 15500 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.181886 11519 rpcz_store.cc:276] 0709 13:16:24.120665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.752427 (+2631762us) service_pool.cc:224] Handling call
0709 13:16:26.753079 (+   652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.753216 (+   137us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:27.009574 (+256358us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.151767 (+142193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.151815 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:27.151861 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.151939 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.152064 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":260473,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":374797,"threads_started":2}
I20260709 13:16:27.153278 15511 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:27.182387 15620 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:16:27.186728 11257 maintenance_manager.cc:419] P ab9dad95b38845f197fb7c3c12a492c4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10591 bytes on disk
I20260709 13:16:27.225214 15667 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:27.225594 15667 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:27.194754 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.226413 11256 maintenance_manager.cc:643] P ab9dad95b38845f197fb7c3c12a492c4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":311,"lbm_reads_lt_1ms":4}
W20260709 13:16:27.226926 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.137156 15246 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.230907 15246 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:27.231383 15643 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.208894 15500 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:27.236441 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 920) took 3116 ms. Trace:
I20260709 13:16:27.236843 11514 rpcz_store.cc:276] 0709 13:16:24.119882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.750897 (+2631015us) service_pool.cc:224] Handling call
0709 13:16:26.807960 (+ 57063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.808626 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.236232 (+427606us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54002a80d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.236376 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124742912}
W20260709 13:16:27.238395 11513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1757) took 3272 ms (client timeout 2000 ms). Trace:
W20260709 13:16:27.238855 11513 rpcz_store.cc:269] 0709 13:16:23.966092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.084112 (+118020us) service_pool.cc:224] Handling call
0709 13:16:27.238181 (+3154069us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400065ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.238324 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3908793600}
W20260709 13:16:27.240103 11366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2359) took 4407 ms (client timeout 2029 ms). Trace:
I20260709 13:16:27.155519 15629 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:27.207651 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1766) took 1711 ms. Trace:
I20260709 13:16:27.243664 11510 rpcz_store.cc:276] 0709 13:16:25.496775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.149980 (+1653205us) service_pool.cc:224] Handling call
0709 13:16:27.179729 (+ 29749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.205151 (+ 25422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.207597 (+  2446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.151362 15656 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 13:16:27.208165 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:27.246351 11368 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2392) took 3096 ms (client timeout 2094 ms). Trace:
W20260709 13:16:27.246977 11368 rpcz_store.cc:269] 0709 13:16:24.150161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.398757 (+248596us) service_pool.cc:224] Handling call
0709 13:16:27.246047 (+2847290us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b54000c6554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.246246 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166306560}
I20260709 13:16:27.248472 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 985) took 2028 ms. Trace:
I20260709 13:16:27.249476 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 998) took 1156 ms. Trace:
I20260709 13:16:27.250025 11368 rpcz_store.cc:276] 0709 13:16:26.092591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.248196 (+1155605us) service_pool.cc:224] Handling call
0709 13:16:27.248633 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.249041 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:27.249074 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.249142 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:27.249182 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.249252 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.249390 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.252079 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1752) took 3309 ms. Trace:
I20260709 13:16:27.252609 11512 rpcz_store.cc:276] 0709 13:16:23.943049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.192161 (+2249112us) service_pool.cc:224] Handling call
0709 13:16:26.721482 (+529321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.745791 (+ 24309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.251830 (+506039us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54003d8e50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.252010 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1216422272}
I20260709 13:16:27.155149 15582 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.181895 15611 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.215907 15665 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.149422 15652 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.217779 15465 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.220288 12061 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:27.224113 15513 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:27.202733 15581 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:27.254289 11368 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:16:27.240514 11366 rpcz_store.cc:269] 0709 13:16:22.832382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.775716 (+943334us) service_pool.cc:224] Handling call
0709 13:16:27.239844 (+3464128us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400166054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.240017 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2823032576}
W20260709 13:16:27.258201 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.248946 11370 rpcz_store.cc:276] 0709 13:16:25.220355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.860368 (+640013us) service_pool.cc:224] Handling call
0709 13:16:27.246826 (+1386458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.247041 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.247068 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.247109 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:27.247149 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.247213 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.248254 (+  1041us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000c6550. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.248398 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3188018432}
I20260709 13:16:27.266795 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:27.276374 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:27.279217 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1862) took 1017 ms. Trace:
I20260709 13:16:27.292160 11445 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:27.297243 11368 rpcz_store.cc:276] 0709 13:16:26.262192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.251169 (+988977us) service_pool.cc:224] Handling call
0709 13:16:27.253900 (+  2731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.278913 (+ 25013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.279152 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:27.299567 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:27.301010 15721 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.081s	user 0.002s	sys 0.000s
W20260709 13:16:27.301581 15703 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.648s	user 0.007s	sys 0.006s
I20260709 13:16:27.302732 15362 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:27.297441 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:27.306124 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2380) took 1680 ms. Trace:
I20260709 13:16:27.308306 11444 rpcz_store.cc:276] 0709 13:16:25.626128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.129887 (+503759us) service_pool.cc:224] Handling call
0709 13:16:26.155883 (+ 25996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.170641 (+ 14758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.170675 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.218756 (+ 48081us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.364966 (+146210us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:26.438801 (+ 73835us) log.cc:844] Serialized 14 byte log entry
0709 13:16:26.639591 (+200790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.639670 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.760979 (+121309us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b18001c7320. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:27.153662 (+392683us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:27.305933 (+152271us) raft_consensus.cc:1685] finished
0709 13:16:27.305954 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.306070 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":73568,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":893,"raft.queue_time_us":3182,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":6460,"spinlock_wait_cycles":70058752,"thread_start_us":88703,"threads_started":1,"wal-append.queue_time_us":367320}
I20260709 13:16:27.296072 12063 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:27.318768 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:27.301523 15701 log.cc:927] Time spent T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.655s	user 0.005s	sys 0.000s
W20260709 13:16:27.276201 15720 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.135s	user 0.000s	sys 0.003s
W20260709 13:16:27.291749 15718 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.093s	user 0.001s	sys 0.001s
W20260709 13:16:27.291594 15713 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.159s	user 0.001s	sys 0.002s
W20260709 13:16:27.291370 15719 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.145s	user 0.002s	sys 0.001s
W20260709 13:16:27.341428 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 162209us
W20260709 13:16:27.292186 15716 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.183s	user 0.003s	sys 0.000s
W20260709 13:16:27.343462 11333 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.100s (SENT)
I20260709 13:16:27.277127 15213 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.276319 15629 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:27.278597 15486 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.226322 15607 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:27.291491 15717 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.150s	user 0.002s	sys 0.000s
I20260709 13:16:27.226323 15667 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:27.360201 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.357998 11370 raft_consensus.cc:1217] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[7.7-10.9]   Dedup: 7.7->[8.8-10.9]
I20260709 13:16:27.361963 11370 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [7.7-10.9]
I20260709 13:16:27.348417 15486 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.372579 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2395) took 1441 ms. Trace:
I20260709 13:16:27.373051 11362 rpcz_store.cc:276] 0709 13:16:25.931228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.121021 (+189793us) service_pool.cc:224] Handling call
0709 13:16:26.121433 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.126035 (+  4602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.126066 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.158092 (+ 32026us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.429191 (+271099us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:26.429238 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.429314 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.239328 (+810014us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:27.372255 (+132927us) raft_consensus.cc:1685] finished
0709 13:16:27.372279 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.372509 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1142,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6146,"spinlock_wait_cycles":39943040,"thread_start_us":184662,"threads_started":1,"wal-append.queue_time_us":431117}
I20260709 13:16:27.370038 15667 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:27.377574 15486 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:27.385957 15581 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.394742 11442 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:27.395831 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:27.396588 11444 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:27.396713 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.398073 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:27.407891 11440 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:27.405402 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:27.404328 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.350021 15213 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.414348 11436 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:16:27.415390 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:27.403537 15536 consensus_meta.cc:287] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 13:16:27.419278 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:27.420644 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.402729 15362 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:27.430789 15223 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.415203 15656 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:27.437958 12069 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:16:27.440040 12060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 791) took 2271 ms (client timeout 2958 ms). Trace:
I20260709 13:16:27.414966 15606 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:27.440708 11335 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.459564 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2260) took 3337 ms. Trace:
I20260709 13:16:27.418116 15465 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:27.463241 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:27.460686 12060 rpcz_store.cc:269] 0709 13:16:25.165219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.225606 (+ 60387us) service_pool.cc:224] Handling call
0709 13:16:27.434225 (+2208619us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54007783d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.436047 (+  1822us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865470848}
I20260709 13:16:27.424226 15593 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:27.436169 15667 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:27.464483 11516 rpcz_store.cc:276] 0709 13:16:24.121813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.809173 (+2687360us) service_pool.cc:224] Handling call
0709 13:16:26.809481 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:26.809752 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.809768 (+    16us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:26.831426 (+ 21658us) log.cc:844] Serialized 52 byte log entry
0709 13:16:27.003837 (+172411us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:27.003873 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.003944 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.459289 (+455345us) raft_consensus.cc:1685] finished
0709 13:16:27.459307 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.459497 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":21085,"thread_start_us":131843,"threads_started":1,"wal-append.queue_time_us":239890}
I20260709 13:16:27.366593 15629 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.460142 15656 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:27.482234 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.483897 15742 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:27.484436 15742 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:27.484805 15742 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:27.434019 15536 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:27.491094 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 50474.1us
I20260709 13:16:27.462615 15223 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.461922 15362 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.472029 15582 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.477726 15593 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:27.488113 15656 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:27.490516 15465 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:27.492888 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.477946 15667 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:27.510502 15726 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:16:27.510808 12066 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:27.502936 15511 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.516062 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1870) took 1222 ms. Trace:
I20260709 13:16:27.526258 11361 rpcz_store.cc:276] 0709 13:16:26.293275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.303839 (+1010564us) service_pool.cc:224] Handling call
0709 13:16:27.304139 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:27.304477 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.304503 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:27.305019 (+   516us) log.cc:844] Serialized 52 byte log entry
0709 13:16:27.505524 (+200505us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:27.505562 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.505623 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.515829 (+ 10206us) raft_consensus.cc:1685] finished
0709 13:16:27.515850 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.515999 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1028,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":18027,"thread_start_us":199951,"threads_started":1,"wal-append.queue_time_us":201783}
I20260709 13:16:27.517297 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1869) took 1228 ms. Trace:
I20260709 13:16:27.496626 15581 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.528785 15362 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.530054 15246 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.530781 15511 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.531589 15593 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.532207 15362 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:27.528260 11368 rpcz_store.cc:276] 0709 13:16:26.288970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.298428 (+1009458us) service_pool.cc:224] Handling call
0709 13:16:27.298746 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.298967 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:27.489403 (+190436us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.516826 (+ 27423us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.516958 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.517000 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:27.517050 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.517132 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.517238 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187157,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":27725,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":185688,"threads_started":2,"wal-append.queue_time_us":19209}
I20260709 13:16:27.534003 15536 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:27.528676 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:27.536623 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.536623 11361 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:27.540458 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:27.543717 15507 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:27.544756 15667 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.546041 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 988) took 2268 ms. Trace:
W20260709 13:16:27.525722 15729 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:16:27.546438 11363 rpcz_store.cc:276] 0709 13:16:25.277679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.912168 (+634489us) service_pool.cc:224] Handling call
0709 13:16:25.912466 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.912772 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.912797 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.913208 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.230483 (+1317275us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b1800350d40. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:27.412715 (+182232us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:27.412757 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.412824 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.545713 (+132889us) raft_consensus.cc:1685] finished
0709 13:16:27.545739 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.545972 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":7485696,"thread_start_us":182032,"threads_started":1,"wal-append.queue_time_us":1578994}
I20260709 13:16:27.549460 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 994) took 1584 ms. Trace:
I20260709 13:16:27.549607 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:27.553267 11361 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:27.549831 11369 rpcz_store.cc:276] 0709 13:16:25.965477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.099954 (+1134477us) service_pool.cc:224] Handling call
0709 13:16:27.100335 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.100654 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.100682 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.107741 (+  7059us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.307803 (+200062us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:27.307842 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.307908 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.549271 (+241363us) raft_consensus.cc:1685] finished
0709 13:16:27.549291 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.549408 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":212,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7100,"thread_start_us":166735,"threads_started":1,"wal-append.queue_time_us":230190}
I20260709 13:16:27.532836 15725 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:27.557118 11369 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:27.542802 15607 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.546734 15652 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:27.573173 11361 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:16:27.570588 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 77759.1us
I20260709 13:16:27.562304 15634 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:27.562419 15667 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:27.563577 15725 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:27.575104 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.571661 15652 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.572672 15486 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 13:16:27.606994 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.609622 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2390) took 3460 ms. Trace:
I20260709 13:16:27.608686 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:27.608919 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 997) took 1529 ms. Trace:
I20260709 13:16:27.610050 11367 rpcz_store.cc:276] 0709 13:16:24.149630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.615112 (+465482us) service_pool.cc:224] Handling call
0709 13:16:25.427051 (+811939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.446876 (+ 19825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.603804 (+2156928us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54000c7450. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.609555 (+  5751us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866707712}
I20260709 13:16:27.614464 11367 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:16:27.614782 11367 pending_rounds.cc:85] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 8
I20260709 13:16:27.614991 11367 pending_rounds.cc:107] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:16:27.615216 11367 raft_consensus.cc:2889] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:27.611826 11364 rpcz_store.cc:276] 0709 13:16:26.079855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.127643 (+1047788us) service_pool.cc:224] Handling call
0709 13:16:27.128047 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.134296 (+  6249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.134330 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.139740 (+  5410us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.207922 (+ 68182us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:27.207955 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.208011 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.608719 (+400708us) raft_consensus.cc:1685] finished
0709 13:16:27.608740 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.608862 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5771,"raft.queue_time_us":6400,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":19409,"thread_start_us":25228,"threads_started":1,"wal-append.queue_time_us":151513}
I20260709 13:16:27.617394 11364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34564
I20260709 13:16:27.617852 11364 raft_consensus.cc:606] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Received request to transfer leadership
I20260709 13:16:27.618307 11364 raft_consensus.cc:614] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:27.603538 15246 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:27.623190 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.604010 15607 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:27.610580 15652 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:27.592104 15667 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:27.591800 15634 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:27.625262 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 839) took 2325 ms. Trace:
I20260709 13:16:27.626020 15507 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.681834 11439 rpcz_store.cc:276] 0709 13:16:25.300087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.300198 (+   111us) service_pool.cc:224] Handling call
0709 13:16:25.321844 (+ 21646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.493819 (+171975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.625054 (+2131235us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400315bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:27.625205 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370668416}
I20260709 13:16:27.622344 15504 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.687346 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 76240.2us
W20260709 13:16:27.697343 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.702149 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:27.698544 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 89953.3us
I20260709 13:16:27.702373 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:27.703337 11364 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:27.704393 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:27.704991 11367 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:27.698565 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1873) took 1361 ms. Trace:
I20260709 13:16:27.705626 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:27.706578 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:27.707367 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:27.706130 11366 rpcz_store.cc:276] 0709 13:16:26.337102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.339617 (+1002515us) service_pool.cc:224] Handling call
0709 13:16:27.339992 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.340228 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:27.476670 (+136442us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.697819 (+221149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.698191 (+   372us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:27.698237 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.698387 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.698513 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10979,"prepare.queue_time_us":139703,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":25256,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":8325888,"thread_start_us":302191,"threads_started":3,"wal-append.queue_time_us":72182}
I20260709 13:16:27.709172 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.709303 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.709566 11513 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:16:27.710222 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.710397 11519 raft_consensus.cc:2376] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9. Re-sending same reply.
I20260709 13:16:27.710568 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.711647 11510 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:16:27.704044 15507 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.715305 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.713593 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.716293 11369 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:16:27.716403 11511 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:27.716791 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.717201 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.717545 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.718662 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.719693 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.716413 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.720485 11514 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:27.720722 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.705677 15711 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
W20260709 13:16:27.718786 15745 log.cc:927] Time spent T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:16:27.723708 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.724790 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.724975 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1874) took 1041 ms. Trace:
I20260709 13:16:27.701237 15500 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.719731 15507 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.727818 15500 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.727746 11518 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:27.728581 15667 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:27.728479 11519 raft_consensus.cc:2376] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7. Re-sending same reply.
I20260709 13:16:27.729803 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.717890 11511 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:27.728190 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.722967 15465 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:27.717566 11369 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:27.725378 11362 rpcz_store.cc:276] 0709 13:16:26.683937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.374694 (+690757us) service_pool.cc:224] Handling call
0709 13:16:27.412112 (+ 37418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:27.483331 (+ 71219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.483367 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:27.484017 (+   650us) log.cc:844] Serialized 52 byte log entry
0709 13:16:27.524930 (+ 40913us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:27.543230 (+ 18300us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.547997 (+  4767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.548071 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.724775 (+176704us) raft_consensus.cc:1685] finished
0709 13:16:27.724794 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.724921 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3788,"prepare.queue_time_us":31662,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":71196,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":2169,"thread_start_us":108541,"threads_started":2,"wal-append.queue_time_us":79220}
I20260709 13:16:27.730589 11519 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:27.698292 15246 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.735260 11514 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:27.736265 11364 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:27.705420 15504 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:27.728852 15223 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.738377 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 838) took 2438 ms. Trace:
I20260709 13:16:27.626977 15536 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:16:27.704222 15544 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.751690 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.752709 11516 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
W20260709 13:16:27.747008 15747 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:16:27.758019 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.749025 11441 rpcz_store.cc:276] 0709 13:16:25.299779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.300017 (+   238us) service_pool.cc:224] Handling call
0709 13:16:25.300270 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.470055 (+169785us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:25.503689 (+ 33634us) log.cc:844] Serialized 14 byte log entry
0709 13:16:25.617952 (+114263us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:25.618004 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:25.618046 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.738183 (+2120137us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54003133d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:27.738225 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.738326 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":764,"prepare.queue_time_us":85570,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":882,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7480,"spinlock_wait_cycles":389979904,"thread_start_us":144985,"threads_started":2,"wal-append.queue_time_us":300187,"wal-append.run_cpu_time_us":1246,"wal-append.run_wall_time_us":1059503}
W20260709 13:16:27.758946 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.760444 11519 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:27.762192 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.762812 11516 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:27.766584 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1767) took 2244 ms. Trace:
I20260709 13:16:27.766984 11517 rpcz_store.cc:276] 0709 13:16:25.522341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.150326 (+1627985us) service_pool.cc:224] Handling call
0709 13:16:27.180013 (+ 29687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.207832 (+ 27819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.207861 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:27.207903 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:27.207936 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.766376 (+558440us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54000e9f54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:27.766421 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.766531 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1454,"raft.queue_time_us":58373,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":18295,"spinlock_wait_cycles":59835264}
I20260709 13:16:27.767858 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1750) took 3825 ms. Trace:
I20260709 13:16:27.734413 15656 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:27.768337 11515 rpcz_store.cc:276] 0709 13:16:23.942565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.027169 (+2084604us) service_pool.cc:224] Handling call
0709 13:16:27.046101 (+1018932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.057711 (+ 11610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.767689 (+709978us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54002c1d50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.767803 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2306111744}
I20260709 13:16:27.758034 15667 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:27.698060 15725 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.748063 15246 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.773639 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.756556 15544 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.780355 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.782840 15544 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.783176 15725 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:27.783502 11513 raft_consensus.cc:2376] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12. Re-sending same reply.
I20260709 13:16:27.784248 15246 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.785584 15634 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:27.791978 11512 raft_consensus.cc:2410] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:16:27.804947 15748 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:16:27.805135 15756 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:27.805037 15751 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.819984 15756 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:27.821292 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.822283 15757 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:16:27.805167 15743 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:16:27.824354 15756 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:27.825562 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1871) took 1531 ms. Trace:
I20260709 13:16:27.825977 11370 rpcz_store.cc:276] 0709 13:16:26.294243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.307503 (+1013260us) service_pool.cc:224] Handling call
0709 13:16:27.307850 (+   347us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:27.362361 (+ 54511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.825266 (+462905us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b54001e6ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:27.825493 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114808704}
I20260709 13:16:27.803473 15742 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:27.829276 15742 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.831722 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:27.832367 15662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.455s	user 0.005s	sys 0.014s
W20260709 13:16:27.832883 15662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.480s	user 0.005s	sys 0.015s
I20260709 13:16:27.835250 11334 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:27.836510 11368 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:27.820225 15751 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:27.843819 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.846529 15742 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:27.847254 15742 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:27.850301 15742 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.850674 15742 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:27.862519 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:27.862502 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.865072 11513 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:27.866415 11516 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:16:27.866962 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:27.869304 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 135381us
I20260709 13:16:27.869813 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.869328 11513 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:27.871008 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:27.871361 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:27.871618 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:27.872165 11445 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:27.873847 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.874265 11519 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:27.879644 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:27.880095 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:27.880913 11439 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:27.881779 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:27.882529 11444 raft_consensus.cc:2410] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:27.883018 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:27.883662 15405 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:27.884050 12062 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:27.884246 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:27.884358 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:27.885144 15751 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:27.882375 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:27.887167 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.888134 11513 raft_consensus.cc:2410] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:27.803468 15652 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:27.895254 12064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 804) took 1633 ms (client timeout 1560 ms). Trace:
W20260709 13:16:27.896113 12064 rpcz_store.cc:269] 0709 13:16:26.261537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.261682 (+   145us) service_pool.cc:224] Handling call
0709 13:16:27.895077 (+1633395us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54001ff9d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:27.895183 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868794752}
I20260709 13:16:27.902629 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.884045 15405 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:27.911760 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:27.911953 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.912652 11519 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:27.912580 12068 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:27.921056 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.921778 11514 raft_consensus.cc:2376] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10. Re-sending same reply.
I20260709 13:16:27.921661 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:27.979173 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.891747 15487 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:16:27.980154 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:27.903765 15504 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:27.892135 15760 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
W20260709 13:16:27.978111 15716 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:16:27.990440 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.991153 11516 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:27.992461 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.993805 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.997823 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:27.998416 11516 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:27.991737 15656 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:28.009167 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.003446 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:28.014079 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:28.015440 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:28.016052 11519 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:28.018257 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.018971 11516 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:28.018976 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:28.019568 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.020174 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.024726 11333 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.023881 15751 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:28.026247 15751 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:28.026624 15751 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:28.028290 15742 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.030439 15742 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.007829 15549 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.031371 15549 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.035097 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1002) took 1269 ms. Trace:
I20260709 13:16:28.002620 15405 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.035709 11510 raft_consensus.cc:2410] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:28.036108 15762 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.035493 11368 rpcz_store.cc:276] 0709 13:16:26.765457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.535550 (+770093us) service_pool.cc:224] Handling call
0709 13:16:27.535809 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.841495 (+305686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:28.034870 (+193375us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400259254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:28.035037 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679830400}
I20260709 13:16:28.023592 15763 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:28.033351 15652 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:28.038359 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:28.038661 15749 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:27.999557 15643 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:16:28.038964 12064 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.041296 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.039052 15749 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:28.033615 15751 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:28.259820 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.248687 11515 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:16:28.237106 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.241752 11514 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:28.245880 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:28.144920 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.067672 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:28.239992 15223 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:16:28.268432 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.269068 11368 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:28.269621 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1248) took 1584 ms. Trace:
I20260709 13:16:28.213562 15756 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.240074 15092 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:16:28.235423 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:28.275113 11443 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:28.279577 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1074) took 1011 ms. Trace:
I20260709 13:16:28.280084 15667 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:28.280560 15667 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.280640 11443 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:28.106549 15478 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.176s
I20260709 13:16:28.282769 15478 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:28.283118 15478 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:28.292994 15478 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.294138 15478 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:28.155664 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:28.075636 12064 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:28.270040 12069 rpcz_store.cc:276] 0709 13:16:26.685215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.685326 (+   111us) service_pool.cc:224] Handling call
0709 13:16:27.436170 (+750844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.460315 (+ 24145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:28.269438 (+809123us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b54007783d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:28.269563 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1719340928}
I20260709 13:16:28.296288 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.276669 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:28.278913 11444 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:28.270574 15756 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.228883 11440 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.189059 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.298077 11511 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.279964 12066 rpcz_store.cc:276] 0709 13:16:27.268117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.300537 (+ 32420us) service_pool.cc:224] Handling call
0709 13:16:27.300844 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.541568 (+240724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:28.279338 (+737770us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54004a65d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:28.279522 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469955840}
I20260709 13:16:28.298717 11443 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.148146 11367 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:28.239707 11438 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:28.058444 15636 raft_consensus.cc:993] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:28.266801 11517 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:28.186877 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 807) took 1385 ms. Trace:
I20260709 13:16:28.304232 12061 rpcz_store.cc:276] 0709 13:16:26.801389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.802021 (+   632us) service_pool.cc:224] Handling call
0709 13:16:28.186709 (+1384688us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400595bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:28.186817 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953765760}
I20260709 13:16:28.149178 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.162220 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:28.305505 11441 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:28.083495 11363 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:28.146807 15213 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:28.286603 11369 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:28.286989 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.287247 11519 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:28.288491 15751 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:28.171619 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.326784 15725 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:28.328461 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:28.329592 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.330518 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.289697 15749 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.059279 15465 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:28.158324 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:28.338768 11436 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:28.107084 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:28.291360 15362 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:16:28.291507 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 266854us
I20260709 13:16:28.295730 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:28.111923 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 124121us
I20260709 13:16:28.177294 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:28.300033 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:28.340029 11511 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:28.340745 11443 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:28.340099 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.301287 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:28.182895 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 126353us
I20260709 13:16:28.059585 15622 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:28.342593 15622 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:28.342399 15743 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:28.345234 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.347165 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.348634 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:28.063647 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.155460 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:28.307623 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.350028 11363 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:16:28.349606 11439 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
W20260709 13:16:28.309790 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 268576us
I20260709 13:16:28.243063 15605 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:28.306097 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:28.312410 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.312844 15675 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:28.365712 11333 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.313562 15606 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.314699 11369 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:28.315497 15698 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:28.315944 15662 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.373756 15662 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.374663 15598 raft_consensus.cc:993] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:28.375106 15598 raft_consensus.cc:1081] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:28.316155 15092 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.364245 15605 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:28.316241 15430 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:28.316397 15643 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:28.247221 15750 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:28.317513 15636 raft_consensus.cc:1081] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:28.375785 15662 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.376096 15092 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.377218 15750 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:28.343437 15622 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.380223 15092 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:28.380316 15622 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:16:28.151831 15744 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.603s	user 0.000s	sys 0.004s
I20260709 13:16:28.378023 15760 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.381054 15675 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:28.381383 15760 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.320286 15776 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:28.320955 15740 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:28.323421 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.324298 15638 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:28.383682 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 857937fa61dc400a81345944f0a14511 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 01ae50c877d04ba687e4e06259ba5581 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:28.325160 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:28.351336 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 55683.1us
I20260709 13:16:28.033457 15549 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.344218 15593 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.036376 15405 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:28.364181 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:28.350476 15629 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:16:28.368465 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:28.329236 15725 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:28.344312 15465 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:28.363148 15653 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.378432 15643 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:28.319669 15667 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.384091 15638 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:16:28.384841 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 59772.6us
I20260709 13:16:28.380476 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.344172 15246 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.384768 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1000) took 1619 ms. Trace:
I20260709 13:16:28.412384 15549 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:28.464795 11365 rpcz_store.cc:276] 0709 13:16:26.764915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.530482 (+765567us) service_pool.cc:224] Handling call
0709 13:16:27.530749 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:27.530934 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:28.243812 (+712878us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b180042f220. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35bc5a64bc 00007f35b98f448b 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf
0709 13:16:28.243953 (+   141us) log.cc:844] Serialized 14 byte log entry
0709 13:16:28.384467 (+140514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:28.384514 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:28.384557 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:28.384625 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.384719 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34711,"prepare.queue_time_us":36241,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":22222,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3289,"spinlock_wait_cycles":93088256,"thread_start_us":130858,"threads_started":2,"wal-append.queue_time_us":118468}
I20260709 13:16:28.353879 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.466229 15549 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:28.467173 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 113380us
I20260709 13:16:28.468209 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.468432 15549 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.468999 15549 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.398696 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:28.470109 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 71500.7us
I20260709 13:16:28.470881 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.470616 11263 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:28.389500 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:28.365739 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:28.505288 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 115908us
I20260709 13:16:28.328743 15756 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:28.507537 15756 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:28.507901 15756 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:28.508230 15756 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:28.510236 15653 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.510789 15653 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.407188 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:28.512439 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 105322us
I20260709 13:16:28.512949 15653 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:28.373160 15213 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:28.513592 15213 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:28.514554 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:28.514426 15213 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.515820 15213 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:16:28.376819 11268 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39438 (request call id 360) took 18528 ms (client timeout 15000 ms). Trace:
I20260709 13:16:28.518682 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.396984 15362 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:28.515928 15675 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:28.520123 15362 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.397293 15742 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:28.516387 15611 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:28.399127 15430 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.527945 11366 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:28.529527 15549 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:16:28.426566 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 46150.3us
I20260709 13:16:28.531519 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:28.532754 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:28.532982 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 184428us
I20260709 13:16:28.533689 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.534339 11333 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:28.428773 15465 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.535542 15465 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:28.535982 15791 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.536410 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:28.429867 15638 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:28.537252 15794 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.413393 15725 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:28.516292 11268 rpcz_store.cc:269] 0709 13:16:09.848642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.849000 (+   358us) service_pool.cc:224] Handling call
0709 13:16:28.376746 (+18527746us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:21.485263 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:21.587547 (+102284us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:21.587594 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:16:21.587613 (+    19us) tablet.cc:662] Decoding operations
0709 13:16:21.589230 (+  1617us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:21.589320 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:21.589345 (+    25us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:16:21.590804 (+  1459us) tablet.cc:701] Row locks acquired
0709 13:16:21.590831 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:16:21.590985 (+   154us) write_op.cc:270] Start()
0709 13:16:21.591112 (+   127us) write_op.cc:276] Timestamp: P: 1783602981590920 usec, L: 0
0709 13:16:21.591144 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:16:21.593274 (+  2130us) log.cc:844] Serialized 7786 byte log entry
0709 13:16:22.977442 (+1384168us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:23.457391 (+479949us) op_driver.cc:464] REPLICATION: finished
0709 13:16:23.752992 (+295601us) write_op.cc:301] APPLY: starting
0709 13:16:23.775988 (+ 22996us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:24.177373 (+401385us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:24.177421 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:24.204499 (+ 27078us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:24.204850 (+   351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=40
0709 13:16:24.204911 (+    61us) write_op.cc:312] APPLY: finished
0709 13:16:24.205395 (+   484us) log.cc:844] Serialized 139 byte log entry
0709 13:16:24.205582 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:24.205753 (+   171us) write_op.cc:454] Released schema lock
0709 13:16:24.205808 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:24.206218 (+   410us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:24.206247 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":128523,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":255076,"apply.run_cpu_time_us":7151,"apply.run_wall_time_us":476521,"num_ops":20,"prepare.queue_time_us":124161,"prepare.run_cpu_time_us":25030,"prepare.run_wall_time_us":280414,"raft.queue_time_us":426956,"raft.run_cpu_time_us":9002,"raft.run_wall_time_us":125374,"replication_time_us":1866092,"spinlock_wait_cycles":16598272,"thread_start_us":563979,"threads_started":4,"wal-append.queue_time_us":357536,"wal-append.run_cpu_time_us":41391,"wal-append.run_wall_time_us":3251366}]]}
I20260709 13:16:28.436473 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:28.539384 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 103001us
I20260709 13:16:28.437599 15667 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.418678 15776 raft_consensus.cc:2764] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
W20260709 13:16:28.397138 15772 consensus_queue.cc:1175] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34a42aa6aa0a4778a532c6433d70b981" responder_term: 9 status { last_received { term: 8 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:16:28.444808 15790 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:16:28.365195 15504 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:16:28.453418 15750 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.412992 15634 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:28.462467 15786 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:16:28.376713 15605 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:28.547453 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 177080us
I20260709 13:16:28.547178 11263 catalog_manager.cc:5697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 11. New cstate: current_term: 11 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
W20260709 13:16:28.542832 15607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.039s	sys 0.029s
I20260709 13:16:28.377930 15662 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:28.423763 15787 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
I20260709 13:16:28.474303 15711 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:28.424000 15643 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:28.472112 11334 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:28.480847 15593 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.472886 15763 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:28.480257 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:28.476646 15536 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:28.472529 15749 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:28.478358 15789 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 13:16:28.481371 15246 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:28.489962 15223 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.534512 15757 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:28.537252 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:28.591670 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 54496.9us
I20260709 13:16:28.592842 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.594054 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:28.363504 15656 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:16:28.546756 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:28.463447 15751 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:28.556972 15605 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:28.542254 15725 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:28.559548 15750 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.564018 11263 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 11, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 11 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.604730 11263 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 12 to 15, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 15 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.543216 15794 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:28.610340 15794 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.610716 15794 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:28.542529 15740 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:28.545054 15743 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:28.567553 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 95518.6us
W20260709 13:16:28.570039 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 89865.2us
I20260709 13:16:28.612917 11484 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:28.561805 15607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.039s	sys 0.029s
I20260709 13:16:28.568884 15536 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:28.547909 15634 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.565248 15662 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.561401 15788 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:28.574760 15504 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:28.603232 15092 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.591286 15223 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.607686 11263 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 6, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 6 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:28.662467 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 62807.5us
I20260709 13:16:28.613096 11334 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:28.613845 11365 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:16:28.610344 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 70003.4us
W20260709 13:16:28.663878 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50842.4us
W20260709 13:16:28.631634 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 56370.1us
I20260709 13:16:28.612001 15743 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:28.597324 15791 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:28.602739 15653 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:28.601286 15667 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:28.635330 15607 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:28.661633 15634 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:28.663441 11263 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 7 to 11, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 11 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.686080 11263 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:28.666258 11366 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:28.677976 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:28.690186 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.599604 15629 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:16:28.636299 15246 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:28.606125 15636 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:16:28.661406 15662 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:28.696244 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.696846 11511 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.698329 11513 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:28.698702 11519 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:28.699694 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:28.700206 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.632372 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.701582 15606 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:28.690122 15762 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:28.702096 15606 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.702600 15606 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.702183 15762 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:28.703364 15762 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.703739 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.704332 11512 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:28.701687 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 69393.7us
I20260709 13:16:28.706321 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:28.707077 15246 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:28.686995 11263 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:28.708319 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.709051 15246 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:28.709625 15246 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.711308 11511 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:28.711050 15762 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.711026 11512 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:28.711637 11263 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 7 to 10, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 10 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.576408 15638 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.714660 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:28.633482 15740 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:16:28.664404 15504 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.662523 15657 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:28.727221 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:28.735666 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.681193 15743 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:28.736490 11263 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 10, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 10 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.743166 11436 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.681686 15362 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:16:28.749485 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 202792us
I20260709 13:16:28.753968 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.756753 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:28.725263 15615 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.720779 15593 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:28.758641 11263 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 4 to 5, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 5 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.711781 15246 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.769912 15756 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.806623 15246 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:28.722019 15223 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.806936 11263 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 7 to 11, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:28.808797 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:28.808698 11263 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 9, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:28.713831 15763 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 13:16:28.810293 11263 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 7 to 9, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 9 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.762681 15662 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.741488 15504 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:28.714164 15711 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:16:28.776517 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:28.794147 15607 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:28.805285 15796 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:28.809368 11439 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:28.745292 15536 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:28.827739 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:28.812613 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 61292us
I20260709 13:16:28.830330 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.831086 11512 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.745720 15656 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.749475 15762 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.838508 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:28.858173 11513 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:28.840536 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:28.852262 11511 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.859266 11514 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:28.855890 11365 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.860632 11514 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:16:28.860999 11514 pending_rounds.cc:85] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 7
I20260709 13:16:28.861208 11514 pending_rounds.cc:107] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:16:28.861421 11514 raft_consensus.cc:2889] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:28.862069 15598 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.862576 15598 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.864539 15636 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:28.864712 15801 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.865044 11514 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48448
I20260709 13:16:28.818265 15740 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.819862 15622 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:28.866397 11514 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:28.879539 11514 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:28.877642 11518 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:28.889673 11512 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:28.805971 15615 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.894834 11512 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:16:28.783303 15606 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:28.904808 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 150910us
I20260709 13:16:28.905777 11484 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.898279 11513 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:28.898890 11514 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:28.899005 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 232084us
I20260709 13:16:28.904338 11518 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48448
W20260709 13:16:28.900429 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.910082 11513 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:28.913516 11514 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.913844 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:28.914813 12068 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:28.888688 15740 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.918115 15803 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:16:28.918982 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:28.919600 12061 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:28.920339 11515 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.904280 15762 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:28.920677 15740 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.923097 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:28.923388 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:28.924959 12062 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:28.888243 15804 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.936956 12063 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:28.933151 12061 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:28.872709 15486 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:16:28.940600 12067 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16. Ops: [14.11-15.12]
I20260709 13:16:28.866215 15656 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.954100 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:28.960626 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:28.950125 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:28.961763 12069 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:28.950502 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:28.950831 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:28.942168 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:28.920256 15606 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:28.896570 15809 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:28.962823 12063 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:28.951355 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:28.960244 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:28.963276 11409 leader_election.cc:336] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.560s (SENT)
I20260709 13:16:28.964352 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.954313 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:28.967382 12065 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:28.968019 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.916368 15756 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:28.966890 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 114106us
I20260709 13:16:28.961306 12060 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:28.970387 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:28.971771 12063 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:28.972122 11367 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:28.916082 15478 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:16:28.977476 11364 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:28.977905 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:28.978596 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:28.979153 12069 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:28.951288 15740 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.970947 15757 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.989377 15740 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.989512 15757 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:28.873360 15801 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:28.960160 15778 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:28.954856 15430 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.946274 15762 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:28.989544 15815 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:16:29.021886 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:29.022557 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 112479us
I20260709 13:16:28.969373 15803 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:16:28.986469 15598 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.022585 11367 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:28.952056 15804 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.030381 11369 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:28.990440 15799 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:29.008397 15793 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:16:28.987828 15606 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:29.020993 15362 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:29.044502 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 178507us
I20260709 13:16:29.045395 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:29.047367 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 77080.2us
I20260709 13:16:29.048137 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.007133 15652 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:16:28.985846 15798 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:16:29.056217 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:28.999460 15634 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:29.056699 11484 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:29.045634 15803 consensus_queue.cc:1243] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:29.064198 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.025121 15812 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:16:29.021759 15430 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.072594 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.020740 15778 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.052294 15818 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:29.056166 15756 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:29.080468 15817 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:29.083156 11516 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:16:29.109647 11516 pending_rounds.cc:85] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 10
I20260709 13:16:29.109922 11516 pending_rounds.cc:107] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:16:29.110144 11516 raft_consensus.cc:2889] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:29.061760 15806 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:29.077279 15711 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:29.087334 11439 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:29.110857 12065 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:29.102759 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:29.122881 11367 raft_consensus.cc:2410] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:16:29.110921 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.123931 11513 raft_consensus.cc:2410] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:29.078943 15653 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:29.050911 15743 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:29.103613 15807 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:16:29.150331 11333 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:29.089145 15549 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:29.100965 15750 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:29.156960 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:29.156082 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 99965.7us
I20260709 13:16:29.105983 15763 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:29.160416 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 178601us
I20260709 13:16:29.062026 15819 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:16:29.111938 15605 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:29.161355 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.077486 15811 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
W20260709 13:16:29.212642 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47780.1us
I20260709 13:16:29.215307 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.143846 15822 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:29.217670 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.218442 11443 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:29.148530 15653 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:29.213928 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:29.162210 15606 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:29.158815 15549 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:29.168093 15788 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:29.099467 15794 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.155543 15711 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:29.166172 15656 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:29.239003 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88769.9us
I20260709 13:16:29.255033 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:29.243052 15822 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.252470 15778 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.210278 15662 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.215308 15750 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:29.213853 15756 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:29.085111 15643 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.281342 15643 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:29.263248 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 206594us
W20260709 13:16:29.263940 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 154265us
I20260709 13:16:29.238960 15811 consensus_queue.cc:1243] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:29.154822 15826 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.149s
I20260709 13:16:29.242281 15772 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:29.237534 15827 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:29.105105 15818 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:29.250322 15605 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:29.176419 15763 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.264065 15791 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.243361 15808 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:16:29.286720 12069 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:29.129078 15817 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.241238 15652 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:16:29.321493 15829 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:16:29.321796 15652 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:29.322144 15652 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:29.295094 15605 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.323635 15711 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.294879 15092 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:29.324913 15092 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:29.325282 15092 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:29.283345 15822 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.151911 15362 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.286245 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:29.298854 15772 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:29.300331 15825 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:29.300681 15763 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.301004 15845 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:29.285852 15757 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:29.323660 15605 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:29.307632 15606 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.313998 15622 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:29.314599 15740 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.315104 15801 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:29.315834 15832 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:29.316483 15430 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.316725 15743 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.290869 15653 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.343727 15832 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:29.344198 15653 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.344549 15832 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.345741 15832 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:29.347352 15667 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:29.355052 15667 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:29.347468 15662 consensus_queue.cc:1175] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b1906ab37ba541f493936685a808c8a9" responder_term: 10 status { last_received { term: 10 index: 9 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 9 } }
I20260709 13:16:29.347801 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 1027) took 1220 ms. Trace:
I20260709 13:16:29.356496 15772 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.356559 11361 rpcz_store.cc:276] 0709 13:16:28.127476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.136619 (+  9143us) service_pool.cc:224] Handling call
0709 13:16:29.347608 (+1210989us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54001e64d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:29.347742 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731055488}
W20260709 13:16:29.360013 15725 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.000s
I20260709 13:16:29.360433 15725 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:29.362378 15725 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.317365 15818 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:29.319027 15223 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:29.350701 15504 consensus_meta.cc:287] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.000s
I20260709 13:16:29.364187 15504 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:29.324512 15536 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:29.320906 15817 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.368251 15750 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.369025 15638 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:29.369998 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:29.291543 15820 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:16:29.294703 15246 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:29.330405 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44232.2us
W20260709 13:16:29.385274 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.385560 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.389324 15622 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.389905 15622 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.368093 15629 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:29.317440 15634 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.322918 15652 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.307147 15794 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:29.335702 15838 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:29.340821 15615 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.352519 15792 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:16:29.353111 11316 auto_rebalancer.cc:893] Removing replace marker failed: Replica 01ae50c877d04ba687e4e06259ba5581 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:29.421605 11316 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:16:29.328852 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 167566us
I20260709 13:16:29.373008 12064 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:29.382922 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.384002 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.422684 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.315748 15804 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:29.395630 11436 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:29.395503 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:29.403102 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.304152 15791 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:29.421955 11316 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:16:29.404937 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:29.405471 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 118237us
I20260709 13:16:29.435292 11440 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:29.404021 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.406742 11517 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:16:29.423987 11483 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:29.435454 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:29.425669 15799 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:29.401824 15629 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:29.442495 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.436765 12034 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:29.401721 15487 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:29.444407 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49041.4us
I20260709 13:16:29.425722 15504 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.445571 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.448675 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:29.451297 12063 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:29.449301 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.454959 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.455674 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.457885 11436 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:29.459235 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.460251 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.461212 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.461870 11444 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:29.461825 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.421144 15792 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:29.461313 11441 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:16:29.434428 15246 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:29.464258 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.463126 11442 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.463207 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.465080 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.467319 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.468557 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.472196 12066 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:29.467113 12069 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:29.473677 11445 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:16:29.468360 11436 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:29.480479 11442 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:29.450230 15634 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:29.485841 11515 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:16:29.445838 15851 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:29.557540 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:29.558200 12063 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.496433 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:29.529842 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.466691 15487 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.565269 15487 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:29.486255 15622 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.462661 15838 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:16:29.466775 15834 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:16:29.484110 15092 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.580082 11438 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:29.581287 15092 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.583714 15792 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:29.585124 15792 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.586318 12062 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:29.566999 15605 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.587884 11511 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:29.587255 11438 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:29.590677 12061 raft_consensus.cc:2410] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:16:29.593696 15850 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:29.594448 15861 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:29.594877 15861 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.595216 15861 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:29.477059 15849 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:16:29.602814 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.466765 15852 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:29.603539 11440 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:29.554301 15851 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.606000 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:29.606449 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:29.681977 12063 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:29.682821 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:29.683439 12063 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:29.684214 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:29.677994 15430 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:29.684985 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:29.686864 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 233090us
W20260709 13:16:29.689956 11408 leader_election.cc:336] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 2.000s (SENT)
I20260709 13:16:29.687767 12063 raft_consensus.cc:2368] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:29.600988 15634 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:29.691028 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.688045 15834 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:16:29.691982 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:29.693009 15834 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:29.685029 15711 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:29.693522 11515 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:29.718899 15711 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.719638 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:29.720220 11515 raft_consensus.cc:2376] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10. Re-sending same reply.
I20260709 13:16:29.720978 15711 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.721024 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:29.721781 11515 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:29.702003 15813 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:29.703123 11443 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:29.602077 15803 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:16:29.599699 15629 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.704420 12066 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:29.686081 15643 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.605643 15606 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:29.717924 15861 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.688630 15430 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.684638 15851 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:29.736395 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.736923 11515 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:29.723285 15834 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.736347 15430 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.743256 15865 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:29.679447 15092 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:16:29.748378 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:29.703109 15653 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:16:29.755787 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.758132 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:29.733819 15643 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.759315 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.732580 15667 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.764699 11515 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
W20260709 13:16:29.775825 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.753156 15834 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.781607 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.736677 15852 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:16:29.785312 15852 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:29.785678 15852 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:29.785480 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.789333 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:29.790182 11513 raft_consensus.cc:2410] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:16:29.793159 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:29.794770 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.792726 15622 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:16:29.807283 15643 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:29.808128 11513 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:29.762079 15803 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:16:29.808900 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.809577 11440 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:29.810825 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:29.811789 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1029) took 1258 ms. Trace:
I20260709 13:16:29.812171 11368 rpcz_store.cc:276] 0709 13:16:28.553643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.553788 (+   145us) service_pool.cc:224] Handling call
0709 13:16:28.799251 (+245463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:28.961932 (+162681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.961987 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:28.962694 (+   707us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.211722 (+249028us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:29.211757 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.211826 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.283795 (+ 71969us) raft_consensus.cc:1685] finished
0709 13:16:29.283816 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.811524 (+527708us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54000c7450. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:29.811732 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24298,"raft.queue_time_us":71773,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":14357,"spinlock_wait_cycles":563768704,"thread_start_us":345889,"threads_started":2,"wal-append.queue_time_us":303070}
I20260709 13:16:29.812139 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:29.816001 15808 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:29.816428 15808 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.816772 15808 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:29.818241 11510 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:29.820708 11510 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:29.821990 11510 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:29.824575 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:29.824769 15606 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:29.825193 15606 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.825749 11436 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:29.764981 15865 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:16:29.794122 15812 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.805251 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2300) took 1109 ms. Trace:
I20260709 13:16:29.826856 15812 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.827180 11517 rpcz_store.cc:276] 0709 13:16:28.696113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.697818 (+  1705us) service_pool.cc:224] Handling call
0709 13:16:28.698094 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:29.435985 (+737891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.804999 (+369014us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54000668d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:29.805185 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629222656}
I20260709 13:16:29.829969 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:29.830382 15812 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.830722 15611 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:29.831133 15611 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:29.831516 15611 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.832232 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.833336 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:29.834375 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.835024 12067 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:29.839622 15810 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:29.840035 15810 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:29.840845 15810 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.842057 15810 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.845171 11443 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:16:29.845664 11445 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:29.845872 11436 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:29.847597 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.847781 15762 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:29.810753 15824 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.516s	user 0.002s	sys 0.000s
I20260709 13:16:29.849989 11369 raft_consensus.cc:1217] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[7.7-10.9]   Dedup: 7.7->[8.8-10.9]
I20260709 13:16:29.850478 11369 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [7.7-10.9]
I20260709 13:16:29.850829 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.826848 15865 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:29.851711 15865 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:29.852448 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:29.852511 15865 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.853803 15865 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:29.854869 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.855612 12065 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 6: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:29.856392 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:29.856925 12065 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:29.857323 11510 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:29.857937 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.758603 15864 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:29.860688 12068 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:29.860802 11515 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:29.860939 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:29.863705 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:29.865973 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.866519 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.866663 11510 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:29.834942 15430 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.870033 12066 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 11. Ops: []
I20260709 13:16:29.786509 15852 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.875597 12064 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:29.865904 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.761413 15634 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:29.883252 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.883891 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:29.867704 15850 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:29.885984 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.884733 15725 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:29.906813 12062 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:29.885808 15791 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:29.886178 15812 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:16:29.916703 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.917812 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.880149 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.915107 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.918702 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.918937 11445 raft_consensus.cc:2393] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 7: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:29.919566 11513 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:29.914613 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.922322 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:29.910336 15809 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:16:29.928063 11370 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:29.929626 11436 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:16:29.900749 15850 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:29.932772 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:29.934253 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 173826us
I20260709 13:16:29.935667 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:29.937559 11439 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:29.904376 15792 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:29.937914 11408 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.936324 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:29.939787 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.948740 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:29.950603 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:29.949306 11408 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:29.941315 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:29.951551 11370 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:29.949177 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:29.952869 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:29.958401 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:29.958843 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:29.933864 15822 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:29.960578 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:29.963876 11513 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:29.968140 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:29.934104 15762 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:29.966809 15828 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:29.971133 15762 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.946990 15362 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.970805 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:29.973915 11513 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:29.973239 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:29.975176 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:29.977334 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.956293 15749 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:29.978744 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 952) took 1142 ms. Trace:
I20260709 13:16:29.962002 15846 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:29.979329 11518 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:29.965664 15223 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:29.961741 15662 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:29.981736 11518 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:29.956765 15864 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:29.979565 11519 rpcz_store.cc:276] 0709 13:16:28.836520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.836635 (+   115us) service_pool.cc:224] Handling call
0709 13:16:28.877333 (+ 40698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.877539 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:28.897042 (+ 19503us) log.cc:844] Serialized 14 byte log entry
0709 13:16:29.972406 (+1075364us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b18003ffe80. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35bc5a64bc 00007f35b98f448b 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e23a
0709 13:16:29.978260 (+  5854us) log.cc:844] Serialized 14 byte log entry
0709 13:16:29.978417 (+   157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:29.978460 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:29.978508 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.978586 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.978689 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5460,"prepare.queue_time_us":20784,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":10449792,"thread_start_us":51724,"threads_started":1,"wal-append.queue_time_us":77614}
I20260709 13:16:29.917209 15808 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.938464 15751 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:16:29.969502 15805 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:16:29.987412 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7a5787304dfe4697baeaeb367ee7dda3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 01ae50c877d04ba687e4e06259ba5581 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:29.982547 15092 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:29.984982 11369 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:29.951507 15817 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:16:29.977247 15792 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:29.989265 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:29.994757 11370 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:29.966666 15750 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:16:29.965183 15711 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:16:29.983491 15846 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.967154 15725 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:16:29.957379 15791 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:16:30.006549 12069 raft_consensus.cc:2368] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8.
I20260709 13:16:29.994473 15792 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.992583 15812 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:29.979084 15822 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:29.990513 15805 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:29.996096 15430 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:29.981591 15362 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.035655 15838 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:30.036125 15838 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:30.008778 15846 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:30.020932 15792 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:29.995736 15504 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:16:29.994478 15845 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:16:30.029147 15465 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:30.008702 15809 consensus_queue.cc:1059] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:16:29.993942 15611 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.031075 15751 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:30.032020 15812 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:30.052651 11518 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.072688 15504 consensus_queue.cc:1243] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:30.075160 15791 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:16:30.007962 15850 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.072306 15465 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:30.077376 15751 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:30.081128 15865 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:30.138655 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:30.081305 15762 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.159866 15808 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.160612 15812 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.108093 15838 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.166668 15838 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.166877 15611 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.167999 15362 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.150319 15799 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:30.168720 15837 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:30.169283 15837 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:30.170315 15837 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.168743 15465 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.174655 15465 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.159899 15605 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.169219 15799 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.182335 15799 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.185287 15861 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.188637 11367 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:30.191829 15822 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.197227 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.197883 11437 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.200040 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.200623 11444 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.202663 11518 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.203732 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.204295 11439 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.204890 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.207563 15861 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.145517 15865 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:30.190819 15606 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.212638 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.213263 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.232215 15837 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:30.203146 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:48448 (request call id 958) took 1346 ms. Trace:
I20260709 13:16:30.251626 11518 rpcz_store.cc:276] 0709 13:16:28.856775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.880604 (+ 23829us) service_pool.cc:224] Handling call
0709 13:16:30.201483 (+1320879us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002c06d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f5a3d 00007f35c19c5cd0 00007f35baf4a254 00007f35baf4a1e2 00007f35baf4a111 00007f35baf4a08c 00007f35baf49342 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:30.203062 (+  1579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2461329280}
I20260709 13:16:30.214268 11444 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:30.217123 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.223644 11439 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:30.203081 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:30.228385 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.231599 11437 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:30.255918 11438 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:30.257072 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44527.4us
W20260709 13:16:30.259392 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 71456.7us
W20260709 13:16:30.348209 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 385264us
W20260709 13:16:30.351889 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 387813us
I20260709 13:16:30.353219 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:30.269057 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 277967us
W20260709 13:16:30.363682 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.003s	sys 0.010s
W20260709 13:16:30.364187 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.003s	sys 0.010s
W20260709 13:16:30.364552 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 677079us
I20260709 13:16:30.369969 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:30.373865 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:30.145359 15805 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.377220 15605 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:30.276922 15223 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:30.413087 15223 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.297283 15792 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.382803 12064 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:30.231886 15698 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:30.261837 15611 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:30.420053 15845 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:30.420733 15611 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:30.207774 15751 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.278308 15662 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:30.427662 15662 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:30.348462 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 352244us
W20260709 13:16:30.423388 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.254772 15828 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:30.297780 15834 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.375471 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.313445 15846 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.246644 15851 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.465996 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1886) took 1491 ms. Trace:
I20260709 13:16:30.224953 15861 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.470453 15861 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.308496 15864 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.466473 11367 rpcz_store.cc:276] 0709 13:16:28.964976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.151266 (+186290us) service_pool.cc:224] Handling call
0709 13:16:29.155354 (+  4088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.189132 (+1033778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.432047 (+242915us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001e50d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:30.456606 (+ 24559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375486720}
I20260709 13:16:30.350862 11333 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.471800 15851 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.391422 15606 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.397185 15805 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.472612 11443 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:30.474402 15549 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.405313 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.475497 11440 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:30.407609 11409 leader_election.cc:400] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:30.476523 11409 leader_election.cc:403] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:16:30.418109 15888 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:16:30.478556 15851 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:30.479704 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.480440 15606 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.481405 11439 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.485430 11366 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:30.485764 15792 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:30.486570 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.481194 15606 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.420970 15845 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.488612 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:30.489538 11368 raft_consensus.cc:2410] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:16:30.489694 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 139180us
W20260709 13:16:30.481247 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.382076 15762 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:30.419356 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:30.423345 15611 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.428637 15698 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.440915 15883 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.441303 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.441545 15885 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:30.448009 11513 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:16:30.432086 15881 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:30.459244 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 106334us
I20260709 13:16:30.464368 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:30.276113 15865 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:30.477972 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70797us
I20260709 13:16:30.408828 15870 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:30.492210 12066 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:30.486469 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:30.493207 15698 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:30.491957 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:30.498498 15885 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.500742 15813 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:30.501410 15870 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.501915 15870 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.502400 15885 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.495366 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.503679 15652 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.504187 11511 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:30.504987 11518 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.505259 15881 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:30.505740 15881 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:30.506122 15881 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.507467 15667 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.493353 15762 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.501253 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:30.508682 11517 raft_consensus.cc:2376] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10. Re-sending same reply.
I20260709 13:16:30.489882 15845 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.489534 15606 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.513577 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:30.513445 15861 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.517392 11334 leader_election.cc:400] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8.
I20260709 13:16:30.519418 11334 leader_election.cc:403] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:30.517530 15834 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.510800 11437 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:16:30.520208 15861 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.523236 11335 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:30.525619 11362 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:30.526726 11365 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:30.526356 15828 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.501248 15813 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:30.532080 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:30.534179 11517 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.537719 15667 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:30.531657 15878 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.227s	user 0.003s	sys 0.001s
I20260709 13:16:30.537765 15822 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.533164 15549 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.541983 15622 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.542711 15622 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.538079 15884 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.534276 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:30.534741 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.547907 15711 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:16:30.548599 15762 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.548789 15711 consensus_queue.cc:1243] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:30.549423 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:30.550303 11361 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:30.552815 12066 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:30.553580 15813 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.555574 15486 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.556129 15486 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:30.558665 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.551179 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:30.561065 12034 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.562218 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.553131 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 812) took 1628 ms. Trace:
I20260709 13:16:30.568423 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:30.569310 15750 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:30.569767 15750 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:30.555491 12061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 815) took 1629 ms (client timeout 1729 ms). Trace:
I20260709 13:16:30.571943 15629 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.574033 11516 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:16:30.574460 11516 pending_rounds.cc:85] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 6
I20260709 13:16:30.574756 11516 pending_rounds.cc:107] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:16:30.575016 11516 raft_consensus.cc:2889] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.563548 12068 rpcz_store.cc:276] 0709 13:16:28.924450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.966559 (+ 42109us) service_pool.cc:224] Handling call
0709 13:16:29.860481 (+893922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:29.863125 (+  2644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.552730 (+689605us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54005af0d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:30.552944 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030208768}
I20260709 13:16:30.577663 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.579980 11512 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:16:30.574121 15861 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.581220 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.581501 15861 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:30.582185 15861 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:30.582959 15430 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:30.577528 15808 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:30.570843 15750 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.584470 15750 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.582466 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.555230 15813 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:30.592862 11439 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:30.559157 15846 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.594024 11518 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:30.571655 12061 rpcz_store.cc:269] 0709 13:16:28.926208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.941864 (+ 15656us) service_pool.cc:224] Handling call
0709 13:16:30.555092 (+1613228us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b540046d554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:30.555303 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1442889472}
W20260709 13:16:30.601660 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54926.5us
I20260709 13:16:30.602999 15740 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:30.603876 15740 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.606499 15740 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.607776 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.608592 11510 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:30.614542 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.615458 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.615793 11443 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:30.616182 15879 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.312s	user 0.002s	sys 0.000s
W20260709 13:16:30.616410 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48162.2us
W20260709 13:16:30.617684 15873 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 13:16:30.607797 15791 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:30.619951 15791 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:30.620051 11513 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:30.620054 15622 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.621119 15791 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.622071 15622 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:30.622664 15622 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:30.623919 15622 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.625252 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2309) took 1515 ms. Trace:
W20260709 13:16:30.628257 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.615000 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.629756 11437 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:30.614522 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:30.631378 11519 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:30.632309 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:30.632606 11512 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:30.633337 12068 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:30.633731 12068 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:30.632651 11439 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:30.637750 12061 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:30.638610 11517 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:30.641816 15865 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.643524 15740 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.644924 11442 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:30.646595 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 895) took 1162 ms. Trace:
I20260709 13:16:30.634720 12068 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.649154 12068 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.648710 15698 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.665001 15698 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.650390 11408 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:30.650864 15848 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:30.665699 15848 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:30.666649 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:30.666887 15725 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:16:30.666813 15848 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.653194 15852 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.667634 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:30.668462 11518 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:30.668927 15751 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:30.652453 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:30.671289 15848 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.646936 11442 rpcz_store.cc:276] 0709 13:16:29.483970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.496137 (+ 12167us) service_pool.cc:224] Handling call
0709 13:16:30.646377 (+1150240us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002085d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:30.646508 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487851264}
I20260709 13:16:30.663255 12068 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:30.656498 11440 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:30.656922 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:30.625623 11514 rpcz_store.cc:276] 0709 13:16:29.109652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.109795 (+   143us) service_pool.cc:224] Handling call
0709 13:16:29.110169 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:29.299011 (+188842us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:29.299042 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:29.813586 (+514544us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b2c000fe208. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:29.814211 (+   625us) log.cc:844] Serialized 52 byte log entry
0709 13:16:30.077792 (+263581us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:30.099952 (+ 22160us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.173453 (+ 73501us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b180039aae0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf 00007f35bca18038
0709 13:16:30.173683 (+   230us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.173892 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.173971 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.245393 (+ 71422us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b180039aae0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:30.616816 (+371423us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:30.624901 (+  8085us) raft_consensus.cc:1685] finished
0709 13:16:30.624925 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.625099 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38692,"prepare.queue_time_us":166819,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":196484,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":21430,"spinlock_wait_cycles":35605376,"thread_start_us":451217,"threads_started":2,"wal-append.queue_time_us":489957}
W20260709 13:16:30.634645 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 83609.6us
I20260709 13:16:30.667423 15725 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:30.619119 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1789) took 1157 ms. Trace:
I20260709 13:16:30.674135 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:30.675341 15653 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:30.677845 11516 rpcz_store.cc:276] 0709 13:16:29.461797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.461910 (+   113us) service_pool.cc:224] Handling call
0709 13:16:29.462173 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.575563 (+1113390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.618848 (+ 43285us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540031ef54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:30.619034 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2555606400}
I20260709 13:16:30.597782 15861 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.680154 12062 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:30.677330 11409 connection.cc:256] RPC call timeout handler was delayed by -1.007118852000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
I20260709 13:16:30.686542 11366 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:30.676532 15725 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.687543 15634 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:30.688179 15634 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.690575 15634 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.691995 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:30.691131 15634 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.693840 15634 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.694871 11518 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:16:30.696084 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:30.696717 11366 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.697393 11335 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:30.698825 15749 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:30.699828 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.683045 15870 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.702024 11445 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:16:30.704147 15870 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.706198 11367 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:16:30.706276 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:30.707381 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:30.707497 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:30.708493 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:30.712112 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 61857.8us
I20260709 13:16:30.691011 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:30.684590 15810 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:30.691836 15725 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:30.718794 11514 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:30.720031 15884 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8.
I20260709 13:16:30.714728 15810 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.720917 15810 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:30.721604 15486 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:30.722438 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.720065 15634 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.679095 15881 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.727521 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.729103 15881 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.729837 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.731369 11515 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:16:30.731974 15092 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:30.732551 15092 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.734570 15092 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.735774 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:30.737975 11511 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:30.738862 15740 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.736552 11514 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:30.735380 15549 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:30.723850 11409 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:30.746683 11370 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:30.732645 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:30.751845 15667 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.720196 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.748301 12062 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
W20260709 13:16:30.754035 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 171714us
I20260709 13:16:30.754434 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:30.757714 11518 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:30.758350 11366 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:30.752873 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:30.757269 11516 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:30.752084 15870 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:30.737596 15749 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:30.769662 11367 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:30.746914 11511 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:30.759498 15808 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:30.767441 15852 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:30.790813 15852 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.791301 15852 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.792143 15667 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:30.792760 15667 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.793656 15808 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.796458 15667 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.769567 15848 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.797199 15848 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.797871 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:30.797927 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:30.798882 11366 raft_consensus.cc:2410] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:16:30.799047 15465 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:30.761853 15549 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:30.758610 15884 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:30.823312 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:30.768148 15698 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.810289 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:30.807799 15813 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:30.820897 15653 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:30.846827 11516 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.811559 15465 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:16:30.833401 11370 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:30.834839 11445 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:16:30.835414 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.838048 11369 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:30.820338 15430 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:30.837987 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:30.840054 15549 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.843649 15884 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.835870 15667 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:30.850950 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 137942us
I20260709 13:16:30.849151 11445 pending_rounds.cc:85] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 8
W20260709 13:16:30.852046 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 65594.2us
I20260709 13:16:30.853739 11445 pending_rounds.cc:107] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:16:30.854275 11445 raft_consensus.cc:2889] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.858479 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:30.808027 15634 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:16:30.860530 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 101643us
I20260709 13:16:30.861982 15430 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.866430 15465 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:30.866984 15465 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.867509 15465 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:30.833280 15749 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.869516 15749 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.866660 15698 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.874948 11444 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:30.875890 15698 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.876080 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:30.867138 15805 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.876850 15805 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.877609 15792 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:30.878181 15792 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.878491 15845 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:30.878995 15805 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.877645 15881 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
W20260709 13:16:30.866930 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.882968 15792 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.866900 15486 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:30.884321 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.884802 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:30.799345 15848 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:30.866564 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:30.885798 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 133003us
I20260709 13:16:30.886178 11333 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:30.868410 11262 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 5 to 9, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 9 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:30.887418 15846 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:30.887903 15846 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:30.888476 11516 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:30.888586 15817 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:30.888814 15846 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.889232 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:30.876773 11443 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:30.883872 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.890468 11511 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:30.883670 15850 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:30.896857 15850 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:30.897828 15850 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.898772 15850 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.899003 15740 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.899557 15740 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:30.901611 15740 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:30.903723 15881 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:30.904670 15749 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.904317 15881 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:30.863085 15611 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:30.884294 15486 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.893325 11440 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:30.892637 15817 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:30.893563 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:30.913612 11443 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:30.914283 11437 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:16:30.914971 11369 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:30.888119 11262 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 11, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.909816 15486 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:30.917620 15662 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:30.919976 11445 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
W20260709 13:16:30.922374 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.922473 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.920465 11445 pending_rounds.cc:85] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 11
I20260709 13:16:30.925725 11445 pending_rounds.cc:107] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:16:30.925973 11445 raft_consensus.cc:2889] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.919327 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.926471 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:30.909943 15817 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:30.930001 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:30.933205 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 74991us
I20260709 13:16:30.915771 15698 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.938436 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:30.919282 15881 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:30.917703 15870 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:30.977157 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44492.8us
W20260709 13:16:30.977902 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.979269 15882 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:30.973814 15605 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:16:30.981513 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:30.983120 15751 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:30.984329 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:30.984931 11518 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:30.995428 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:30.999354 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 64434.9us
I20260709 13:16:30.985767 15881 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:31.002488 15751 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:31.013797 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 98174.4us
I20260709 13:16:31.015007 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.017349 11369 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
W20260709 13:16:31.018983 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:31.019605 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.015638 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:31.000519 15882 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:31.058280 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45272.1us
I20260709 13:16:30.997893 15870 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.064095 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:31.066870 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54111.1us
I20260709 13:16:31.068370 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:31.069445 11369 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:31.063314 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:31.088747 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.091138 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:31.091113 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:31.012416 15698 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.106904 11442 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:31.107934 12068 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:31.110759 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:31.103593 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:31.117764 15808 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:31.120860 15803 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:31.121385 15803 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:31.121760 15803 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:31.122984 15750 consensus_meta.cc:287] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.003s
I20260709 13:16:31.123711 15882 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.075480 15805 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:31.126041 15805 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.126412 15805 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.133322 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:31.135113 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:31.136502 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:31.137399 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.025501 15881 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:31.148211 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 68755.6us
I20260709 13:16:31.152053 11364 raft_consensus.cc:2410] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:16:31.152463 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.089200 15430 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:31.164450 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.126271 15882 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.140344 15611 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.061950 15653 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:16:31.237180 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:31.075897 15662 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:16:31.246721 11262 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:31.250075 11262 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:31.254045 15906 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 13:16:31.255640 15908 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 13:16:31.257679 15910 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:16:31.263873 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 193592us
I20260709 13:16:31.268682 15870 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.151490 15223 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:31.236390 11367 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:31.151394 15885 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:31.153196 15605 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:31.153460 15749 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:31.291553 15223 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:31.291671 15749 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:31.292233 15749 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:31.292873 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.292609 15223 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.293246 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:31.293759 11440 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:31.242067 15751 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.069983 15810 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.245774 15725 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.245816 11443 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:31.245496 15611 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.243356 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:31.285594 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.286412 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.287029 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.300923 11440 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
W20260709 13:16:31.287289 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 150887us
I20260709 13:16:31.297551 15881 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.298061 15884 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.302870 11365 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:31.305152 15667 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.273494 15848 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.276717 15791 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:31.352576 15848 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:31.285562 15652 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.781s	user 0.008s	sys 0.004s
I20260709 13:16:31.297626 15810 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.322606 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.325752 15817 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.355605 11440 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:31.355938 15848 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.356781 15817 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.336717 15809 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.337000 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:31.338016 15605 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.358971 15848 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.359089 15605 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.359889 15848 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.361218 15605 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.328923 15909 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:31.361900 15848 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.358228 15809 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.357631 15667 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.364681 11518 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:31.365689 15809 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:31.366575 15430 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:31.367623 15430 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.376327 15809 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.377192 15809 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:31.421747 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 131367us
I20260709 13:16:31.428660 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:31.298028 15725 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.323695 15465 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.434602 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.435338 11517 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:31.433267 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.460323 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.425246 15611 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.464308 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:31.465103 11512 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:31.462433 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:31.461248 11514 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:31.343151 15486 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.472648 11370 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:31.424463 15810 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:31.471009 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:31.478529 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.479164 11440 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
W20260709 13:16:31.481235 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 45329.1us
I20260709 13:16:31.430392 15848 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.467540 15605 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:31.486240 15810 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.486649 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.487414 11367 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:31.486972 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.490159 11513 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:16:31.490449 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 423849us
W20260709 13:16:31.478390 15092 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.012s	sys 0.019s
I20260709 13:16:31.484051 15698 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.495671 15931 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:31.497525 12063 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:31.498292 15653 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.518272 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.518520 15653 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.519001 11440 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:31.520694 15653 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:31.501943 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 236330us
I20260709 13:16:31.482542 15611 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:31.522125 15611 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:31.507804 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 173826us
I20260709 13:16:31.508320 15751 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.474725 15846 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:31.522238 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:31.481853 15662 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:31.523766 15931 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:31.541664 11408 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:31.531888 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:31.511821 15092 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.012s	sys 0.020s
I20260709 13:16:31.542762 11515 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:31.548429 11512 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:31.552556 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.532269 15846 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.568907 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:31.582711 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 60587.3us
I20260709 13:16:31.586246 11519 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
W20260709 13:16:31.568954 15852 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.000s
I20260709 13:16:31.590817 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.528024 15848 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.536481 15881 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:16:31.536316 15805 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.544523 15662 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.576313 15846 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:31.539321 15751 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.600008 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.608560 11440 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:31.609061 11444 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:31.581687 15927 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:31.613161 12069 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:31.596239 15809 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.616032 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:31.596664 15852 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.624182 15486 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:31.625494 15935 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:31.626972 15936 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.627686 15936 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.629933 15936 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.630251 15667 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.632043 15848 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.633904 15751 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.636008 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.636471 15662 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.636809 11367 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:31.638372 15223 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:31.638991 15223 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.640674 15805 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:31.641036 15223 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.641889 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:31.642011 15927 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:31.642670 11437 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:31.641894 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.643250 15927 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.647876 15926 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:31.648376 15926 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.648715 15926 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:31.649277 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.650848 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.651427 11367 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:31.648049 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:31.653545 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:31.654232 15929 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:31.654614 15929 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.654922 15929 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.657593 11442 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:31.659577 15223 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:31.660619 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.660562 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.660043 15223 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:31.673305 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:34564 (request call id 11) took 1078 ms. Trace:
I20260709 13:16:31.673430 15223 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.673744 11361 rpcz_store.cc:276] 0709 13:16:30.596254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.600894 (+  4640us) service_pool.cc:224] Handling call
0709 13:16:31.671435 (+1070541us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54003228d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca04735 00007f35bc9fc9fa 00007f35c19c44c9 00007f35baf40c54 00007f35baf40be2 00007f35baf40b11 00007f35baf40a8c 00007f35baf3fd42 00007f35bad3948c 00007f35bad388c6
0709 13:16:31.671636 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588697472}
W20260709 13:16:31.675206 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 857937fa61dc400a81345944f0a14511 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 01ae50c877d04ba687e4e06259ba5581 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:31.663477 15810 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:31.664337 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 122764us
W20260709 13:16:31.665441 15934 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:16:31.664377 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.593884 15931 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.689474 15931 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:31.630632 15725 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.700456 11442 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:31.669602 15749 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.608803 15611 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:31.672158 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 56206.5us
I20260709 13:16:31.682679 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.683710 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:31.666709 15850 consensus_meta.cc:287] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.768s	user 0.008s	sys 0.004s
I20260709 13:16:31.666766 15929 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.712879 11361 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.704152 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:31.705443 11445 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:31.682289 15848 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:31.690502 15653 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.707672 11366 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:31.621527 15605 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:16:31.709774 15749 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.683936 15465 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.726060 11444 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:31.714800 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:31.716703 15653 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.689353 15549 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:16:31.702445 15865 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:31.717845 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 562296us
I20260709 13:16:31.705763 15810 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.718557 15850 raft_consensus.cc:2764] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:16:31.727253 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44518.8us
I20260709 13:16:31.724332 15881 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.740820 11437 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
W20260709 13:16:31.729394 15803 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.003s
I20260709 13:16:31.745811 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.747117 11516 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:31.706072 15813 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:16:31.749006 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:31.715508 15662 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:16:31.750037 15881 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.751729 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:31.752238 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.752688 15653 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.732308 15605 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:31.761253 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 72472.8us
I20260709 13:16:31.761966 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:31.762939 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 61066.6us
I20260709 13:16:31.763253 15949 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:31.764532 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.765125 11365 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:31.744935 15810 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.742414 15865 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.768460 15850 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:31.769649 15870 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:31.770035 15870 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:31.770120 15865 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:31.771982 12063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1261) took 2281 ms (client timeout 2404 ms). Trace:
I20260709 13:16:31.768389 11518 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:31.774017 15808 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:31.774859 15808 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:31.796693 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:31.772388 12063 rpcz_store.cc:269] 0709 13:16:29.490832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.691776 (+200944us) service_pool.cc:224] Handling call
0709 13:16:31.771792 (+2080016us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540034b554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:31.771928 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3775485184}
I20260709 13:16:31.801256 11441 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:31.763778 15949 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:31.767241 15605 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.804783 15605 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:31.805612 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 85740.1us
I20260709 13:16:31.806830 11335 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:31.804608 15949 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:31.809135 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:31.811262 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 111215us
I20260709 13:16:31.770884 15870 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.813925 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:31.772114 15760 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.814574 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.751132 15803 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.775235 15808 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:31.819252 11518 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:31.809377 15850 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.832795 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:31.833938 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:31.836220 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:31.839433 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:31.840371 11437 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:31.841926 15611 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.842514 15611 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.845180 15611 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:31.851999 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 103062us
I20260709 13:16:31.817723 15653 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:31.853749 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44696.2us
I20260709 13:16:31.864507 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.867553 11438 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:31.869532 15948 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.869693 11511 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.870136 15948 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.872243 15948 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:31.866221 15653 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:31.879648 15882 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:31.869722 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.882488 11514 raft_consensus.cc:2410] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:31.852922 15751 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:31.888038 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 80352.7us
I20260709 13:16:31.860013 15813 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.890352 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:31.865339 15465 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:31.853180 15749 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:31.896265 15749 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.896701 15749 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:31.897166 11437 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:31.896683 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.901192 11518 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:31.828761 15549 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:16:31.880986 15870 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:31.900839 11515 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:16:31.902029 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56077.2us
I20260709 13:16:31.905097 11361 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:16:31.906803 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.853093 15667 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:16:31.880774 15929 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:31.908075 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.860513 15865 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:16:31.911931 11511 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:31.912434 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:31.884963 15955 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:31.913528 11361 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:31.880300 15653 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.860154 15850 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.887748 15751 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.914218 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.933162 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.933468 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:31.933947 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.934275 11362 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:31.934026 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.938699 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.939332 11365 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:31.933050 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.926350 15955 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.936092 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.941823 11370 raft_consensus.cc:2410] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:31.908357 15808 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.939769 11514 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:31.827032 15760 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:31.942888 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.941377 11368 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:31.938781 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.961597 11515 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:31.907871 15941 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:31.927269 15817 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.900110 15605 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:31.939082 15751 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.969586 11518 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:16:31.971958 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:31.974212 11511 raft_consensus.cc:2410] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:31.978045 11515 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:16:31.942348 15955 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:31.978843 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:31.943140 15958 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:31.935160 15813 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.986305 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:31.985278 11514 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:31.991559 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:31.994273 11518 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:16:31.976408 15653 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:31.995779 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:31.983351 15808 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:31.998252 11515 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.006125 11445 raft_consensus.cc:2410] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:16:32.007967 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.004766 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:31.974140 15817 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:31.995672 15549 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.018260 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:32.025305 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.026950 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:32.032152 15956 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:16:32.033712 15947 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:16:32.034708 15634 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:32.035727 15605 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:32.009176 15848 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:31.991036 15846 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.060678 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.061043 11363 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:32.062247 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.069036 11362 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:32.003230 15430 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:16:31.997876 15926 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.027906 15929 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.004462 15611 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:16:32.091034 15817 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.078161 15611 consensus_queue.cc:1243] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:32.109279 15846 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.110198 15881 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.092116 15760 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.096294 15813 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.045113 15549 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.003531 15958 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.115921 15549 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.116674 15958 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:32.118104 15951 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.239s	user 0.000s	sys 0.002s
I20260709 13:16:32.078542 15929 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.119521 15810 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.120083 15929 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.123795 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.124431 11517 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:32.124707 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.125451 11513 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:32.127761 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.128971 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.129653 11519 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:32.131358 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.131752 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.132011 12066 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:32.132483 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.098322 15926 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.134757 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.100679 15749 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.133733 12063 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:32.101701 15957 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:32.136222 12065 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:16:32.136480 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.137065 11368 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:32.137190 12061 raft_consensus.cc:2368] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:32.139122 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.102391 15606 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:32.139961 15606 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:32.140589 12033 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.102831 15963 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:32.141076 15606 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.142051 15963 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.143168 15606 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:32.104180 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.144287 11370 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.144691 15963 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.145576 15611 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:32.146034 15611 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.146420 15611 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.104617 15850 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:32.106079 15852 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:32.117955 15949 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.148836 11437 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:32.149245 11437 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:32.150022 15949 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:32.149956 11437 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.152216 15861 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:32.152626 15861 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:32.153506 15861 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.154866 15861 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:32.089639 15223 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.157426 15223 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.133621 15725 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:32.133893 12069 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:32.158592 15725 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:32.159585 15810 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.160069 15810 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.162185 15810 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.161736 15955 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.163515 15749 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.139863 11362 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:32.164810 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.165494 11362 raft_consensus.cc:2368] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:32.166405 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.168502 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.169340 12065 raft_consensus.cc:2410] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:16:32.171527 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 110947us
I20260709 13:16:32.172145 11515 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
W20260709 13:16:32.177693 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.180133 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:32.180701 11443 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.170058 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.183092 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:32.189353 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:32.197556 12066 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
W20260709 13:16:32.202273 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.195577 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:32.195538 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:32.226346 11444 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:32.227265 11515 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:16:32.228163 12061 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:16:32.228540 12061 pending_rounds.cc:85] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 8
I20260709 13:16:32.228742 12061 pending_rounds.cc:107] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:16:32.228955 12061 raft_consensus.cc:2889] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:32.234974 11370 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:32.241250 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.243203 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.243867 12066 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:32.245353 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:32.227152 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:32.247313 11444 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:16:32.245972 11442 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:32.251709 12069 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:16:32.252724 11370 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 13:16:32.253502 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 267262us
I20260709 13:16:32.258168 11443 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:32.175791 15926 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:32.258567 11370 pending_rounds.cc:85] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 8
I20260709 13:16:32.183611 15810 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:32.262713 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.264915 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.222908 15949 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:32.267202 11364 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:32.270078 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.263983 11370 pending_rounds.cc:107] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:16:32.264348 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:32.244616 11483 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 2.404s (SENT) [suppressed 20 similar messages]
I20260709 13:16:32.271811 11370 raft_consensus.cc:2889] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:32.266460 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.225893 15803 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.253691 15852 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:32.262919 15931 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.570s	user 0.004s	sys 0.006s
I20260709 13:16:32.276796 12065 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 6.
I20260709 13:16:32.260008 15958 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:32.271178 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:32.301908 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52395.7us
I20260709 13:16:32.159621 15725 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.326151 11440 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:32.328188 11442 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:32.329841 12067 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:32.273326 15949 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.327093 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.279471 15852 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:32.332702 11408 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:32.303221 11515 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:32.268841 15751 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.338692 12066 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:32.341187 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.341887 11512 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
W20260709 13:16:32.342504 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.343763 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.342207 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:32.275705 15810 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.271551 15926 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.358934 12066 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:32.338904 15949 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.359511 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.339818 15852 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:32.361861 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.395774 12060 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:32.396750 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.397789 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.398728 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.399853 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.403501 11362 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:32.403450 15940 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.404157 15940 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.406646 15940 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:32.363497 12066 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:16:32.407637 15971 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:16:32.407881 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.363644 12064 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:32.364331 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.365535 15926 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.363466 15751 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.367023 15903 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:32.367767 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.367046 11444 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:32.364149 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.387326 15949 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.389778 15611 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.391201 15870 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:32.392735 15970 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.422614 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.423218 11370 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
W20260709 13:16:32.393115 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 50980.5us
I20260709 13:16:32.424315 11409 leader_election.cc:400] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:32.366591 11369 pending_rounds.cc:85] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 9
W20260709 13:16:32.414063 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 64722.3us
I20260709 13:16:32.419724 12067 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:32.440155 11369 pending_rounds.cc:107] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:16:32.440454 11369 raft_consensus.cc:2889] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:32.349248 15803 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:32.350185 15966 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:16:32.411291 15848 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:32.425626 15970 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.425920 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.448020 11512 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.441444 11262 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:32.425989 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.449404 12066 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:32.428593 15949 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.419687 15903 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.428591 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.420974 15870 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.406771 11362 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:32.410094 15955 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.454834 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.430831 11361 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:32.430510 15865 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:32.440016 11409 leader_election.cc:403] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:16:32.439249 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.436733 15605 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:32.441238 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.450135 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.456585 12067 raft_consensus.cc:2368] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
W20260709 13:16:32.457662 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 213055us
I20260709 13:16:32.458503 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.463330 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:32.463621 11333 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:32.465329 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:32.465965 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.469456 15808 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:32.469965 15808 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.470333 15808 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:32.471514 11333 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.487118 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.488830 11517 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:32.489084 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:32.490623 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.490854 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:32.493366 15653 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:32.493906 15653 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.494648 15970 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.495200 15970 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:32.494264 12066 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:32.454309 15870 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.492624 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.496902 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.500887 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:32.497102 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.519672 11370 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:32.467000 15751 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:32.521204 15430 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:32.523068 15974 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:16:32.521345 15751 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.514211 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:32.467563 15662 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.468953 15940 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:32.525522 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:32.564625 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 50489.4us
I20260709 13:16:32.471380 15927 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:32.528115 11361 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:32.526897 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.451895 15848 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.524133 15828 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:32.487730 15223 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:32.498458 15936 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:32.497655 15926 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:32.567243 15940 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.579833 15940 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.564890 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:32.504221 15928 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:32.583076 15928 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:32.567358 15927 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.583817 15927 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.518302 15606 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:32.470794 15605 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.471007 15865 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.495771 15970 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.569002 15430 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:32.585255 11512 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:32.585920 11370 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:32.524983 15846 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:32.493525 15965 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:16:32.590431 11515 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
W20260709 13:16:32.593538 15969 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:16:32.594705 12060 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:32.568785 15881 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:32.595443 15611 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:32.595798 15813 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.596706 15605 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.597518 15940 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.597993 15653 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.572763 15223 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:32.597432 11519 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:32.600997 11364 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:32.601009 15870 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:32.605165 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.607272 15846 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:32.566665 15634 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:16:32.597669 15606 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.614599 15881 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:32.588676 15903 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.588469 15949 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:32.594916 15952 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.722s	user 0.001s	sys 0.003s
I20260709 13:16:32.606006 15430 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.583818 15928 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 13:16:32.620204 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.604887 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.631942 15940 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.636164 15430 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.622556 15223 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.625795 15846 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:32.606285 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 108757us
I20260709 13:16:32.641708 11409 leader_election.cc:400] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:32.641993 11409 leader_election.cc:403] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:32.647615 11437 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.649044 15964 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 8. Current term is 9.
I20260709 13:16:32.633507 15881 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:32.643361 15972 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:16:32.656087 11437 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:32.625577 15813 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.659248 15808 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.653669 15964 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:16:32.665614 15808 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.651978 15903 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.666602 15903 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.668792 15903 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.692305 15751 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.695079 15870 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.697022 15852 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.698067 15865 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.698309 11516 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:32.698450 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:32.701301 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7a5787304dfe4697baeaeb367ee7dda3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 01ae50c877d04ba687e4e06259ba5581 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:32.697713 15760 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.712699 15760 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.714919 15760 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.717000 15813 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.695670 15970 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.749624 15970 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.750203 15970 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.752805 15970 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:32.754818 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 55466.7us
I20260709 13:16:32.756089 15928 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:32.756498 15662 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:32.756633 15928 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:32.651347 15430 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.757663 15928 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.758154 15808 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:32.759235 15430 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.719035 15223 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:32.762491 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 107794us
I20260709 13:16:32.763509 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:32.764773 15970 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:32.765058 15813 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:32.765363 15865 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:32.766073 15970 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:32.767048 15970 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.769336 15881 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.771991 15881 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.755383 15927 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.806463 11439 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:32.778584 15961 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:16:32.751987 15846 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.779920 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1903) took 1309 ms. Trace:
I20260709 13:16:32.779604 15936 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:32.782789 11369 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:16:32.782931 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 200654us
I20260709 13:16:32.813670 12032 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:32.787497 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 218330us
I20260709 13:16:32.778548 15606 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.778502 15223 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.808068 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.817571 11368 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:32.810588 11366 rpcz_store.cc:276] 0709 13:16:31.470364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.470486 (+   122us) service_pool.cc:224] Handling call
0709 13:16:31.489721 (+ 19235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:31.715896 (+226175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:32.779713 (+1063817us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400166a54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:32.779865 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500835200}
I20260709 13:16:32.823173 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:32.819669 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:32.827188 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.827050 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:32.831828 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.831179 11513 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:32.834775 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.836397 11409 leader_election.cc:400] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:32.843744 11513 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.845420 11361 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:32.845717 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.803985 15926 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:32.850766 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.851068 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.851476 12065 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:32.851665 11512 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.845811 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:32.848706 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.853060 11514 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:32.824677 15430 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:32.848178 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.827693 15760 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:32.853978 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.858227 11368 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.856329 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.830329 15606 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:32.861193 12068 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:32.827836 15881 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:16:32.871810 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.872713 15988 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:32.832618 15634 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:16:32.874274 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.872511 12069 raft_consensus.cc:2368] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:32.844969 15987 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:16:32.883571 15936 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.884577 15958 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:32.885056 15958 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:32.888806 12033 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:32.890020 15430 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:32.890623 15430 consensus_queue.cc:1243] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:32.890192 15611 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:32.892268 15846 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.892230 15611 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.893219 15611 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.894279 15751 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.894966 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.894795 15751 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.895713 12063 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:32.896174 15988 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:32.896551 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.897408 15751 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.897164 15988 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:32.897902 12063 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:16:32.898799 11364 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:32.858059 15760 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.894419 15881 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.900072 15760 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:32.900192 15881 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.899153 12063 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:16:32.900761 15611 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.902405 15881 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.904037 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.904420 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.904690 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.905366 15852 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.906257 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:32.905505 11370 raft_consensus.cc:2410] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:16:32.906944 11364 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:32.903926 15810 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.909644 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.914660 11514 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:32.922789 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:32.923022 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 65560.9us
I20260709 13:16:32.923965 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.924482 11364 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:32.927831 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.928187 11512 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:32.928484 12068 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.931849 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:32.932521 11370 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:16:32.934579 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.944655 11364 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:32.945401 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.946801 11514 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:32.946889 12033 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:32.948366 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.948948 11515 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:32.953171 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:32.953929 15852 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:32.954751 15985 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:16:32.926963 15861 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:16:32.957005 15611 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:32.963007 11364 raft_consensus.cc:1217] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[7.7-11.10]   Dedup: 7.7->[8.8-11.10]
I20260709 13:16:32.963572 11364 pending_rounds.cc:85] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 7
I20260709 13:16:32.963826 11364 pending_rounds.cc:107] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:16:32.964088 11364 raft_consensus.cc:2889] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:32.969472 12068 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:32.971310 11513 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:32.971580 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:32.976413 15852 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:32.976742 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:32.976441 15861 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:32.977370 11513 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:32.977721 15861 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.979722 15861 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:32.980882 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:32.981518 12067 raft_consensus.cc:2376] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11. Re-sending same reply.
I20260709 13:16:32.987699 11512 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:32.987665 15958 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:32.989626 11370 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:32.990886 12065 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 7.
I20260709 13:16:32.989411 11368 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:32.992923 12033 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:32.938333 15927 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:32.975553 15903 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:33.079335 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:33.080463 11514 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:33.081811 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.088853 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:32.990080 15958 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.090659 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:33.051738 15852 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.048303 15928 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:33.098711 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:33.100592 11333 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:33.077790 15961 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.103305 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2447) took 1188 ms. Trace:
I20260709 13:16:33.105047 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2445) took 1189 ms. Trace:
I20260709 13:16:33.104562 11363 rpcz_store.cc:276] 0709 13:16:31.911074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.911180 (+   106us) service_pool.cc:224] Handling call
0709 13:16:33.099634 (+1188454us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400259754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:33.099747 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276529920}
I20260709 13:16:33.101452 15740 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:33.107277 11367 rpcz_store.cc:276] 0709 13:16:31.910358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.910464 (+   106us) service_pool.cc:224] Handling call
0709 13:16:33.098535 (+1188071us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b5800003c98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:16:33.099534 (+   999us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5913344}
I20260709 13:16:33.090914 15903 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:33.108938 15928 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:33.114692 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:33.119982 11514 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:33.122938 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:33.090101 15760 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.125605 15988 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:33.126252 15988 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:33.126696 15760 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.127074 15988 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.127485 15927 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:33.129040 15928 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.109160 15740 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.103415 15852 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.138809 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:33.141325 11514 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:33.107225 15961 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.137418 15740 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:33.148965 11409 leader_election.cc:400] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:33.149329 11409 leader_election.cc:403] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:16:33.150051 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:33.150833 15953 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:33.151557 15927 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:33.151214 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:33.152235 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:33.162673 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:33.093645 15926 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.171972 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:33.173827 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:33.141479 15885 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:33.174504 11514 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:33.174717 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:33.144979 15958 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.175067 15611 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:33.176903 15989 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:33.161412 15808 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:33.175478 15995 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:16:33.187879 15927 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:33.188493 15970 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:33.189817 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:33.175335 15606 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:33.176074 15909 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:33.199151 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:33.202494 11514 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:33.203878 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:33.197515 15808 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.204725 15927 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:33.205157 15927 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:33.205790 15881 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:33.206071 15808 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:33.206478 15970 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:33.206883 15970 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:33.207780 15970 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.208989 15970 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.206614 15808 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:33.211732 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:33.145820 15936 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:16:33.182695 15611 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:33.205964 15927 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.195194 15740 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.184027 15958 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.213574 15865 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:16:33.228999 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.220471 15808 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:33.220055 15611 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:33.215653 15606 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:33.220114 15885 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:33.218989 15881 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:33.276180 15881 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.279717 15223 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.280236 15223 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.280625 15740 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:33.281687 15808 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.282331 15223 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:33.285624 15997 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:16:33.215577 15909 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:33.264959 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 90302.4us
I20260709 13:16:33.288242 15909 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.266450 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:33.292485 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:33.293394 15549 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:33.294574 15549 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:33.265280 15865 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:33.267387 15998 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:16:33.181898 15903 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.288969 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:33.226636 15882 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:16:33.272246 16002 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:16:33.299101 11408 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:33.243887 15927 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:33.307614 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 165621us
I20260709 13:16:33.315935 15865 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:33.315833 15982 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.845s	user 0.001s	sys 0.004s
I20260709 13:16:33.317322 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:33.309230 15882 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:33.317902 15999 log.cc:927] Time spent T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:16:33.318198 15634 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:33.318722 15634 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.318070 11518 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:33.317245 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:33.323031 11517 raft_consensus.cc:2393] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:33.327375 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.332196 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:33.333235 11511 raft_consensus.cc:2368] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:33.308852 15662 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:33.342798 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.349329 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.350327 11437 raft_consensus.cc:2410] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:16:33.350409 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.351222 11443 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:33.355654 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.356462 11437 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:33.305755 15223 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:16:33.374066 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.369588 11518 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:33.377133 11441 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:33.370709 11443 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:33.372134 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.326674 15813 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.327214 15852 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.378875 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.379331 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.381538 15961 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:33.382006 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.341711 15953 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:33.390547 11333 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:33.383136 16001 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:16:33.381719 15882 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.389575 11436 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:33.333644 15989 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:33.397150 15817 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:33.397782 15817 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:33.397403 15989 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.404062 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.405972 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.406682 11437 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:33.409350 11513 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:33.379355 15740 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:16:33.413647 11513 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:33.399521 16010 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.418928 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:33.420234 11513 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:33.425402 11513 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:33.426285 11514 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.435716 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.435820 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.440239 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.440898 11444 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:33.391749 15813 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.441794 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.458654 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.460125 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.460417 11438 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:33.438465 15885 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:33.470363 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.472504 11514 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:33.473898 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.474501 11514 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:33.390913 15606 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.476244 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.475291 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.484423 12066 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:16:33.467761 15958 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:33.435423 16010 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.493928 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.497089 11514 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:16:33.498157 11514 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:33.499243 11514 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:33.500589 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.473491 15662 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:33.501951 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.507644 16012 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:33.509011 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.509706 11438 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:33.512828 11445 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:33.514225 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:33.518599 12067 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:33.464285 15817 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.536278 11436 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:33.540153 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.540889 11445 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:33.494220 15808 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:16:33.543855 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.552376 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.555817 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.556602 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.557052 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.557942 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:33.558686 12068 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 9: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:33.563776 11436 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:33.567983 11445 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:33.568542 11438 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:33.567417 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.579236 11442 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:33.579649 11442 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:33.577149 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:33.582149 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.583830 11442 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.595422 11442 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.595319 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.604316 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.609586 15848 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.611039 11442 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:16:33.612188 11442 pending_rounds.cc:85] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 6
I20260709 13:16:33.612648 11442 pending_rounds.cc:107] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:16:33.612957 11442 raft_consensus.cc:2889] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:33.623808 16016 log.cc:927] Time spent T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:16:33.632939 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:33.633384 16014 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:16:33.633805 11442 raft_consensus.cc:2410] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:16:33.634689 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.635039 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1283) took 1269 ms. Trace:
I20260709 13:16:33.635373 12060 rpcz_store.cc:276] 0709 13:16:32.365495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.399632 (+ 34137us) service_pool.cc:224] Handling call
0709 13:16:33.634880 (+1235248us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54004a6fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:33.634986 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446750080}
I20260709 13:16:33.633684 12061 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:33.637183 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1273) took 1371 ms. Trace:
I20260709 13:16:33.637487 12061 rpcz_store.cc:276] 0709 13:16:32.265451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.633082 (+1367631us) service_pool.cc:224] Handling call
0709 13:16:33.633390 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.636954 (+  3564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.637133 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:33.638048 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.639165 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:33.639673 11440 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.644250 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.644969 11442 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:33.645105 11438 raft_consensus.cc:2410] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:16:33.637200 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.650492 11444 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:16:33.655181 11444 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:33.665930 11440 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:33.581815 15611 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.671288 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:33.675527 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:33.676740 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:33.681535 11335 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:33.678587 15953 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:33.683821 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:33.715548 11440 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
W20260709 13:16:33.714813 12064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1272) took 1472 ms (client timeout 1715 ms). Trace:
I20260709 13:16:33.717708 12065 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:33.690295 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:33.585008 15662 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:33.716400 12064 rpcz_store.cc:269] 0709 13:16:32.242587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.242736 (+   149us) service_pool.cc:224] Handling call
0709 13:16:33.714638 (+1471902us) spinlock_profiling.cc:243] Waited 95.7 ms on lock 0x7b54001faed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:33.714758 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220126976}
I20260709 13:16:33.673123 15634 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.723717 11510 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:33.717412 15953 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.725447 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:33.726641 11516 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2353) took 1469 ms (client timeout 1860 ms). Trace:
I20260709 13:16:33.693800 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:33.727313 11516 rpcz_store.cc:269] 0709 13:16:32.253111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.264143 (+ 11032us) service_pool.cc:224] Handling call
0709 13:16:33.722437 (+1458294us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54003d8954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:33.722597 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979657984}
I20260709 13:16:33.682783 15760 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:33.681059 15808 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:33.678624 15882 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:33.717841 15740 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:33.770450 16022 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:16:33.682734 16012 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
W20260709 13:16:33.776002 16021 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:16:33.726387 16010 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.723378 15901 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:16:33.726824 16007 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
W20260709 13:16:33.787040 16023 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:16:33.788529 15611 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:33.768767 15760 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:33.791594 15760 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:33.796213 11516 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:33.797765 11516 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:33.674952 15223 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:33.723229 15430 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:33.799861 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:33.824131 11516 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:33.682214 15813 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:33.821101 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 100574us
I20260709 13:16:33.818626 15882 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:33.827299 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.830613 15882 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:33.831166 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.837736 12061 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:33.799530 15740 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.845666 12034 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:33.826136 15223 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.767818 15926 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:16:33.848507 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:33.856839 16007 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:33.844671 15430 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:33.857555 16007 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.843014 15740 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.858167 15430 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.839116 15909 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:16:33.862709 16007 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:33.863322 11516 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:33.866549 15901 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:33.876518 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:33.886652 16026 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:33.888090 16025 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.212s	user 0.004s	sys 0.003s
I20260709 13:16:33.887652 15740 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.837128 15611 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:16:33.890182 15092 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:33.890475 15740 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:33.875300 15927 consensus_meta.cc:287] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.001s
I20260709 13:16:33.900364 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:33.901693 12061 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:33.852155 15903 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:33.901302 15953 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.901628 15611 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:33.916847 15611 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:33.917760 15611 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.920336 15740 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:33.920794 15740 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:33.921155 15740 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.922462 11442 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:16:33.847432 15928 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:16:33.923792 11444 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:33.924181 11444 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:33.924844 11444 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.925696 11444 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:33.901798 15698 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:33.926118 15698 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:33.926971 15698 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.928134 15698 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:33.887261 16026 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.911185 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:33.851701 15949 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:33.910548 11436 raft_consensus.cc:2410] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 13:16:33.910982 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.912130 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:33.909227 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 857937fa61dc400a81345944f0a14511 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:16:33.935793 11443 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.880908 15813 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:33.933972 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.942466 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:33.944600 15223 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:33.946003 11443 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:33.949923 15882 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:33.994100 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.952701 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 992) took 1454 ms. Trace:
I20260709 13:16:33.954638 11515 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:33.955078 15961 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:33.959180 12033 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:33.963344 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:34.027734 12068 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:33.964162 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:34.030328 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 66244.8us
I20260709 13:16:33.906082 15760 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:33.983105 11516 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:33.951316 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:33.996237 11510 rpcz_store.cc:276] 0709 13:16:32.502295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.525108 (+ 22813us) service_pool.cc:224] Handling call
0709 13:16:33.722691 (+1197583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.724136 (+  1445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.952531 (+228395us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003d8950. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:33.952647 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2664222080}
I20260709 13:16:33.993669 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:34.025811 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:33.983682 12069 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:34.050438 12062 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
W20260709 13:16:34.051765 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 92658.2us
I20260709 13:16:34.048514 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1084) took 1086 ms. Trace:
I20260709 13:16:34.026000 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:34.055066 11444 raft_consensus.cc:2368] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:34.053647 11368 rpcz_store.cc:276] 0709 13:16:32.961935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.050079 (+ 88144us) service_pool.cc:224] Handling call
0709 13:16:33.050398 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.053692 (+  3294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.053721 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.054198 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.793281 (+739083us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b18001c5c40. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:33.969726 (+176445us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:33.969753 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.969815 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:33.969853 (+    38us) raft_consensus.cc:1685] finished
0709 13:16:33.969864 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.048445 (+ 78581us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2922,"raft.queue_time_us":3025,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":10466,"spinlock_wait_cycles":2805632,"thread_start_us":176300,"threads_started":1,"wal-append.queue_time_us":869386}
I20260709 13:16:33.964609 15949 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:34.072788 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:34.075380 11444 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:16:34.075759 11515 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:33.982666 15813 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.982141 15987 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:33.981835 15927 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:33.995482 15092 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:34.049325 15903 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:34.069928 16034 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:34.090966 12068 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
W20260709 13:16:34.098017 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.098364 11515 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:34.027273 15961 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:34.071154 15662 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:34.093710 15813 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:34.108428 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.069796 15953 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:16:34.109992 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.140354 11408 leader_election.cc:400] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:34.105218 16026 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.141384 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:34.107962 15092 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:34.100178 15927 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:34.144088 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 90921.9us
I20260709 13:16:34.108809 15662 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.169628 15662 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.171149 16053 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:34.110260 16044 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:16:34.185488 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 44188.4us
I20260709 13:16:34.201169 11408 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.208139 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:34.182935 16036 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:34.216197 16053 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.217032 15882 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.216801 16053 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.217775 15092 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.218986 16053 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.227413 16036 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.229879 16036 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:34.212208 15953 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:16:34.233603 16049 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:16:34.232347 15882 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.152117 15961 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.242553 15760 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:34.244130 15953 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:34.276793 16065 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:34.279479 11334 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.279644 15662 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:34.399760 15760 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:34.386174 15927 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.387951 11438 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:34.248526 15882 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:34.398231 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 118863us
I20260709 13:16:34.398813 16058 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.419497 15953 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.421674 16058 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.424028 16058 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.405154 16065 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.426673 16065 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.434818 11445 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:34.439527 15813 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.430451 15740 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.414964 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:34.445879 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 956155us
W20260709 13:16:34.447538 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 954103us
I20260709 13:16:34.448393 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:34.449465 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.449777 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:34.409835 12060 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:34.442225 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1000) took 1324 ms. Trace:
I20260709 13:16:34.454386 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.441850 15813 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.460266 11438 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:34.451545 11511 rpcz_store.cc:276] 0709 13:16:33.117793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.393989 (+276196us) service_pool.cc:224] Handling call
0709 13:16:33.413712 (+ 19723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.413983 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:33.432082 (+ 18099us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.441416 (+1009334us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b180037ffc0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:16:34.441769 (+   353us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.441913 (+   144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:34.441956 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:34.442009 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.442069 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.442169 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25336,"prepare.queue_time_us":2040,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":1089,"raft.queue_time_us":1016890,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":6453504,"wal-append.queue_time_us":27399}
I20260709 13:16:34.470707 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.471733 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.451731 16050 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:34.474367 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:34.475946 11334 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:34.529829 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 53964.9us
I20260709 13:16:34.531021 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:34.531701 11369 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:34.530565 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.532413 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.533023 11368 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:34.562379 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.563144 11364 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:34.470594 15634 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:34.516410 15810 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:34.584159 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.468142 15850 raft_consensus.cc:2764] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:16:34.469899 15848 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.586127 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.529027 15698 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:34.558794 15817 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:34.472390 16065 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.587632 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:34.601274 11362 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:34.564602 16036 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:34.515774 15760 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:34.606040 11368 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:34.462250 15881 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
W20260709 13:16:34.632781 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 183061us
I20260709 13:16:34.600764 16050 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:34.634171 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 159876us
I20260709 13:16:34.648700 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:34.649600 11365 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:34.602942 15698 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.668247 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1834) took 1338 ms. Trace:
I20260709 13:16:34.604375 15848 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:34.659880 16066 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.254s	user 0.002s	sys 0.000s
I20260709 13:16:34.664031 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 918) took 2485 ms. Trace:
I20260709 13:16:34.635296 15634 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:34.699836 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:34.606782 15961 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:34.689013 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.695613 11439 rpcz_store.cc:276] 0709 13:16:32.178670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.194067 (+ 15397us) service_pool.cc:224] Handling call
0709 13:16:32.670852 (+476785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.806893 (+136041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.199630 (+1392737us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540027abd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:34.663766 (+464136us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b540027abd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:34.663901 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274837632}
I20260709 13:16:34.611662 15850 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:34.635160 15810 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:34.667173 11511 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:16:34.701386 11368 raft_consensus.cc:2368] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:34.682040 15549 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:34.713625 11365 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:34.712231 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:34.717357 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.718489 11408 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:34.679785 11513 rpcz_store.cc:276] 0709 13:16:33.330156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.435512 (+105356us) service_pool.cc:224] Handling call
0709 13:16:33.435836 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.436081 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:33.459009 (+ 22928us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.646867 (+1187858us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.667904 (+ 21037us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:34.667965 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:34.668017 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.668082 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.668191 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":946941,"prepare.queue_time_us":26018,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":947991,"spinlock_wait_cycles":45185920,"thread_start_us":246469,"threads_started":2,"wal-append.queue_time_us":125840}
I20260709 13:16:34.602893 15817 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:34.702638 15949 consensus_meta.cc:287] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.000s
I20260709 13:16:34.703851 15882 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.704739 15698 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:34.706195 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 175703us
I20260709 13:16:34.738083 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.739429 11334 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.740720 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.604579 16065 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.705962 15634 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:34.724669 16010 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:34.735579 15092 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:34.711763 15885 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:16:34.755470 15850 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.764218 15634 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:34.786567 11513 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:16:34.780740 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:34.789057 11515 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:34.790176 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:34.793227 11367 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:34.787225 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:34.796449 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:34.738111 15813 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:16:34.706642 16050 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.791931 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:34.786966 11513 pending_rounds.cc:85] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 9
I20260709 13:16:34.799913 11513 pending_rounds.cc:107] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:16:34.800194 11513 raft_consensus.cc:2889] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:34.772837 15881 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.764457 15882 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:34.767580 16060 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:34.741231 15949 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.791282 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:34.773629 15092 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.778082 15958 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.822592 11515 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:34.825173 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:34.825727 11515 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:34.826364 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:34.785697 11369 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:34.776986 15549 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:34.812341 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:34.831262 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:34.831282 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.805367 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:34.822023 12033 leader_election.cc:400] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:34.827549 11361 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:34.795868 15698 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:34.829564 11408 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:34.777189 16077 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:34.831997 11513 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:34.841638 11518 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:34.833318 12033 leader_election.cc:403] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:34.804517 16010 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:34.828269 15092 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:34.786811 15850 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.858265 11362 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:34.860563 11515 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:34.902070 11364 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:34.831593 15606 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:34.837564 15810 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:34.914513 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.863847 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.915695 11366 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:34.915123 11362 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
W20260709 13:16:34.847615 16071 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:16:34.913199 11513 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:34.830703 15430 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:16:34.845275 16010 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:34.920842 16085 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:16:34.845357 16082 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:34.845412 15813 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:34.820688 15881 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.953908 15881 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:34.864166 16073 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:16:34.955561 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1005) took 1444 ms. Trace:
I20260709 13:16:34.955926 11514 rpcz_store.cc:276] 0709 13:16:33.510833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.518904 (+  8071us) service_pool.cc:224] Handling call
0709 13:16:33.519196 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.450034 (+930838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.450062 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.450488 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.611963 (+161475us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:34.612002 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.612063 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:34.955321 (+343258us) raft_consensus.cc:1685] finished
0709 13:16:34.955342 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.955493 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":930479,"raft.queue_time_us":942417,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":48003,"thread_start_us":161167,"threads_started":1,"wal-append.queue_time_us":273507}
I20260709 13:16:34.865046 15817 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:34.945079 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:34.958150 11362 raft_consensus.cc:2368] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:34.959005 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.960130 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.960779 11362 raft_consensus.cc:2368] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:34.961521 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.933259 11408 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:34.963817 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:34.965046 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:34.965816 15813 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.831715 15958 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:34.968011 15813 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:34.968732 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:34.862951 16077 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:34.879247 15549 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:34.865074 15987 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:34.836447 16079 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:16:34.845698 15698 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.989733 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:34.909157 15606 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:34.844519 16063 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:16:34.921209 15662 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:16:34.988133 15987 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:35.017688 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2369) took 2087 ms. Trace:
I20260709 13:16:35.018113 11512 rpcz_store.cc:276] 0709 13:16:32.930455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.209276 (+278821us) service_pool.cc:224] Handling call
0709 13:16:33.209622 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:33.296768 (+ 87146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.296800 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:33.309225 (+ 12425us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.476676 (+167451us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:33.476716 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.012585 (+1535869us) spinlock_profiling.cc:243] Waited 86.8 ms on lock 0x7b5400064fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:35.012643 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.012694 (+    51us) raft_consensus.cc:1685] finished
0709 13:16:35.012709 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.017622 (+  4913us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5416,"spinlock_wait_cycles":199654784,"thread_start_us":162167,"threads_started":1,"wal-append.queue_time_us":234467,"wal-append.run_cpu_time_us":1697,"wal-append.run_wall_time_us":1086326}
I20260709 13:16:35.020992 16054 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 13:16:35.021837 15223 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:35.022234 16054 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:35.022637 16054 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:35.023478 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1131) took 1090 ms. Trace:
I20260709 13:16:35.023833 12060 rpcz_store.cc:276] 0709 13:16:33.933302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.933761 (+   459us) service_pool.cc:224] Handling call
0709 13:16:35.023292 (+1089531us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54005afad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:35.023395 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466174208}
I20260709 13:16:35.024326 16068 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:35.024745 16068 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:35.025405 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2378) took 1071 ms. Trace:
I20260709 13:16:35.025667 16068 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:34.992873 15848 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:34.991842 11362 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:34.944310 16010 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:34.908686 16007 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:16:34.869351 15903 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.027141 16068 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:35.039441 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 946) took 1076 ms. Trace:
W20260709 13:16:35.091827 11441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2456) took 1672 ms (client timeout 2221 ms). Trace:
I20260709 13:16:35.091080 11442 rpcz_store.cc:276] 0709 13:16:33.964636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.281104 (+316468us) service_pool.cc:224] Handling call
0709 13:16:34.384834 (+103730us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.385206 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.385234 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.385681 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.584134 (+198453us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:34.584169 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.584934 (+   765us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.039219 (+454285us) raft_consensus.cc:1685] finished
0709 13:16:35.039239 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.039365 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":73,"thread_start_us":198108,"threads_started":1,"wal-append.queue_time_us":651888}
I20260709 13:16:35.071240 11518 rpcz_store.cc:276] 0709 13:16:33.954296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.469731 (+515435us) service_pool.cc:224] Handling call
0709 13:16:34.535082 (+ 65351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.842082 (+307000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.025132 (+183050us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54000645d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:35.025338 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704582528}
I20260709 13:16:35.028643 16078 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:35.094076 16078 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.096287 16078 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.098726 15813 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:35.092255 11441 rpcz_store.cc:269] 0709 13:16:33.419332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.435603 (+ 16271us) service_pool.cc:224] Handling call
0709 13:16:35.091620 (+1656017us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b540027a1d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:35.091763 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350541056}
I20260709 13:16:34.995939 15606 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:35.089259 15958 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.103085 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1006) took 1543 ms. Trace:
I20260709 13:16:35.108111 11362 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:35.103504 11519 rpcz_store.cc:276] 0709 13:16:33.559452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.559594 (+   142us) service_pool.cc:224] Handling call
0709 13:16:33.567793 (+  8199us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:34.764217 (+1196424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.764250 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:34.772696 (+  8446us) log.cc:844] Serialized 52 byte log entry
0709 13:16:34.833562 (+ 60866us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:34.924417 (+ 90855us) log.cc:844] Serialized 14 byte log entry
0709 13:16:35.009128 (+ 84711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.009205 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.102298 (+ 93093us) raft_consensus.cc:1685] finished
0709 13:16:35.102319 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.103018 (+   699us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":883285,"prepare.queue_time_us":92947,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":1096051,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":15388,"thread_start_us":387030,"threads_started":3,"wal-append.queue_time_us":98431}
I20260709 13:16:35.036296 16050 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.090260 16079 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.029390 16075 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.031392 16087 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:35.022235 15223 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:35.117565 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.033891 16044 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
W20260709 13:16:35.102238 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 169069us
I20260709 13:16:35.120374 16079 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.123713 11408 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:35.110428 15817 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.028988 16048 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:16:35.129812 11367 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:16:35.137794 15698 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.052822 15549 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.124891 16075 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.153350 16010 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.137396 15662 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:35.027760 16077 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.062059 16054 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:35.170320 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 57418.5us
I20260709 13:16:35.130849 15817 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.127619 16087 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:35.169694 16035 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.267s	user 0.003s	sys 0.009s
I20260709 13:16:35.164129 16097 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.165725 16063 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:16:35.194597 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.203800 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:34.861312 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:35.204339 11262 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 9, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:35.206815 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.202857 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.208212 15430 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.208714 15430 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.185315 16054 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:35.210821 15430 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.213619 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:35.213805 15949 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.214991 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.215473 12064 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:16:35.215940 11436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 933) took 1622 ms (client timeout 1762 ms). Trace:
I20260709 13:16:35.216313 12061 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:16:35.219518 16083 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.204s	user 0.001s	sys 0.003s
I20260709 13:16:35.220335 16010 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.221443 16010 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:35.216236 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.223114 16050 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:35.223515 16087 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:35.224304 16024 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.506s	user 0.000s	sys 0.010s
I20260709 13:16:35.225046 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:35.225962 16051 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:35.226446 16051 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:35.215093 12062 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:35.228922 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.231268 12068 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
W20260709 13:16:35.229445 11436 rpcz_store.cc:269] 0709 13:16:33.593294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.603722 (+ 10428us) service_pool.cc:224] Handling call
0709 13:16:35.215759 (+1612037us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400315bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:35.215886 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700222208}
I20260709 13:16:35.238704 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.198016 15852 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.251443 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.242463 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.245344 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:35.253249 12062 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:16:35.246052 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.246315 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.249720 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:35.240389 16060 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:16:35.254688 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:35.254875 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.255486 12067 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:35.256045 12061 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:35.193800 15092 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.249912 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:35.196149 15989 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.257094 15989 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.200438 16063 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:35.264078 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.264520 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.268777 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.185328 16099 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.227335 16051 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:35.269969 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 146321us
I20260709 13:16:35.223536 16050 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:35.266710 12062 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:35.273495 11514 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:16:35.275997 11515 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:35.275935 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.276281 11514 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:35.276981 11511 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:35.233032 15861 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:35.256100 16090 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:16:35.172216 15223 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:35.268268 15961 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.561s	user 0.017s	sys 0.000s
I20260709 13:16:35.284274 11516 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:35.289746 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.259262 15989 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.236575 15850 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.200217 15634 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.288210 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.287744 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.294533 15634 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.278239 16098 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.287642 16079 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.295531 16098 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.296099 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:35.296628 15813 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.297652 16098 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.297775 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:35.298769 15813 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:35.299050 16065 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.299758 16050 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.301036 16050 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:35.184348 16097 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.294486 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.302214 11369 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:35.302707 11517 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:16:35.292328 16075 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.289994 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.302842 11362 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:35.250362 15662 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:35.291924 11365 raft_consensus.cc:2410] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:16:35.302829 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:35.309175 11441 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:35.309432 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:35.309762 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 103727us
I20260709 13:16:35.310725 11367 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:35.316802 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:35.319092 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:35.319892 11443 raft_consensus.cc:2368] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16.
I20260709 13:16:35.289521 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.324270 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.325731 11369 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:16:35.326270 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:35.324208 11518 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:35.326728 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.329521 11514 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:35.330615 11511 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:16:35.324898 11441 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:35.333866 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:35.334319 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.332687 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.337227 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:35.337983 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.338430 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:35.338699 11442 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:35.336776 16074 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.341753 16074 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.342344 16097 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.346072 11442 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:35.346432 11442 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:35.299255 15813 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.278805 15882 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:16:35.349758 15813 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.349942 16074 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.263578 15987 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.351832 15882 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:35.352229 15882 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:35.353056 15882 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.354180 15882 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.323638 15961 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.355701 15961 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.356680 15903 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:35.357463 15903 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.357573 15961 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.357925 15903 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.324805 15223 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.360106 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.362112 11442 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.366248 11442 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:35.332561 16075 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.278129 15852 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.289965 15861 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.330459 15662 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.340000 15940 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:16:35.387646 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.284361 16099 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.396461 11512 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:35.388362 11436 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:35.374262 15909 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.395628 12060 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:35.400836 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:35.403354 11409 connection.cc:256] RPC call timeout handler was delayed by -1.2253094289999353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:16:35.403877 11409 connection.cc:256] RPC call timeout handler was delayed by -1.1947094289998859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:16:35.404223 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.404801 11515 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:35.406618 11365 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:35.378454 15813 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:35.408722 11442 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:16:35.410465 11442 pending_rounds.cc:85] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 6
I20260709 13:16:35.410713 11442 pending_rounds.cc:107] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:16:35.410939 11442 raft_consensus.cc:2889] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:35.411000 11409 connection.cc:256] RPC call timeout handler was delayed by -1.121809428999768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 13:16:35.367154 16097 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:16:35.360006 15989 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:16:35.409003 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 64014.6us
I20260709 13:16:35.424156 11445 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:16:35.424587 11445 pending_rounds.cc:85] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 8
I20260709 13:16:35.424803 11445 pending_rounds.cc:107] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:16:35.425043 11445 raft_consensus.cc:2889] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:35.375733 16075 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.434003 16097 consensus_queue.cc:1243] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:35.448199 16103 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:35.448761 16103 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.449713 15740 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.450699 16103 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.439106 15092 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:35.451332 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:35.484496 11409 connection.cc:256] RPC call timeout handler was delayed by -1.0273094289998426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 13:16:35.486713 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.371691 16063 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.413789 15909 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.489216 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.505110 12063 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:35.508421 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.509127 11444 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:35.510450 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.496152 15606 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.519886 15606 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.522970 15909 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.529973 11441 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:35.532371 15662 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.531535 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:35.523458 15903 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.525609 15852 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.539801 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.541841 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.622134 11441 raft_consensus.cc:2410] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:16:35.615826 16068 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.002s
I20260709 13:16:35.627977 11365 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:35.635679 11441 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:16:35.636219 16101 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:35.636878 16123 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:16:35.669284 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 135958us
I20260709 13:16:35.647636 16099 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.673332 16077 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.655889 16075 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:35.673888 16077 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.680857 11442 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:35.688426 11442 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:35.689960 11444 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:35.690702 11442 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.698093 11442 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.701867 11443 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:16:35.707435 15941 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:35.707685 16051 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:35.708892 15941 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:35.709264 16051 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:35.709431 15740 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:16:35.710176 15970 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:16:35.710968 16051 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:35.712751 16075 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.712941 16096 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:35.713330 16075 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.713436 16096 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.714537 15987 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.716328 16075 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.718402 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.722671 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.722747 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.723503 11519 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:35.723801 11513 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:35.690871 11440 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:35.716352 16096 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:35.695151 16077 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:35.691859 16121 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:16:35.727018 16102 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.472s	user 0.001s	sys 0.006s
I20260709 13:16:35.688323 16104 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:35.729123 11442 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 11. (term mismatch)
I20260709 13:16:35.729523 11442 pending_rounds.cc:85] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 10
I20260709 13:16:35.729863 11442 pending_rounds.cc:107] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:16:35.730154 11442 raft_consensus.cc:2889] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:35.733122 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.733708 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 948) took 1666 ms. Trace:
I20260709 13:16:35.734068 11445 rpcz_store.cc:276] 0709 13:16:34.067053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.435621 (+368568us) service_pool.cc:224] Handling call
0709 13:16:35.216386 (+780765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.425456 (+209070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.732910 (+307454us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400315bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:35.733525 (+   615us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b5400315bd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:35.733635 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271925888}
I20260709 13:16:35.735073 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:35.736111 15961 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:16:35.734503 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.735355 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.743005 11445 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:35.745260 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.745849 11445 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 16: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:35.735678 11443 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:35.749154 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:35.752911 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.753662 15606 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:35.754182 15606 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:35.729506 16104 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:35.756948 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.756918 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.760761 11442 raft_consensus.cc:2410] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:16:35.758778 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:35.725950 15817 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.753679 11436 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:35.751735 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:35.763054 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:35.750417 16096 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:35.765406 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:35.765884 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.767845 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.769542 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.750553 15852 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:16:35.770843 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:35.771159 11443 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:35.751000 16077 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.761533 16104 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:35.760900 16050 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.764305 15740 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:35.775985 11370 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:35.766607 15885 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:35.774922 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:35.770413 15549 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.755052 15606 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.771543 16098 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:35.774752 16082 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:35.783703 16116 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.176s	user 0.002s	sys 0.000s
I20260709 13:16:35.782115 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:35.783555 11367 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:35.771169 16096 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.797804 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.798619 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.799280 11368 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:35.800294 15850 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:35.801151 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:35.802513 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:35.803246 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.803560 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.803999 11369 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:35.804358 11363 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:16:35.806299 15958 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:35.806699 15958 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.807118 15958 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.807730 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:35.808481 11515 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:35.788375 15885 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:35.811973 16074 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:35.812177 15885 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.822992 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.779170 16050 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.799427 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.823796 11513 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:35.776489 16077 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.825935 11367 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:35.828727 11333 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:35.832261 11444 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:35.821036 16082 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:35.835608 11370 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:35.837121 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:35.837569 11367 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:35.840867 16050 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.842751 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:35.846932 16096 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:35.848169 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.843381 11515 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:35.852404 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:35.852772 11364 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:35.853065 11441 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:35.854193 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.855862 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:35.857847 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:35.858317 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.858945 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.859370 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.860057 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.862823 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.866629 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.870959 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.871589 11365 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
W20260709 13:16:35.872335 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.872202 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.829241 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:35.875869 11368 raft_consensus.cc:606] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:35.876324 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.860307 15861 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.877945 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:35.883975 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.885490 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.886188 11364 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:35.887269 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:35.887782 16137 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:16:35.875092 15881 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:35.884691 11365 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:35.876296 11368 raft_consensus.cc:614] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:35.889246 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:35.884347 11335 leader_election.cc:400] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16.
I20260709 13:16:35.894981 11364 raft_consensus.cc:2368] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:35.892930 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1094) took 1219 ms. Trace:
I20260709 13:16:35.895416 11335 leader_election.cc:403] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:16:35.895633 11361 rpcz_store.cc:276] 0709 13:16:34.673483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.673611 (+   128us) service_pool.cc:224] Handling call
0709 13:16:34.827370 (+153759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.833560 (+  6190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.892711 (+1059151us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001e5fd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:35.892865 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352916736}
I20260709 13:16:35.898720 15881 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.916451 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.900693 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:35.929801 15953 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:35.930383 15953 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.940550 11368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:35.942649 11368 raft_consensus.cc:606] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:35.942967 11368 raft_consensus.cc:614] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:35.932523 11335 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:35.898537 11441 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:35.931715 15810 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:35.947094 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:35.947731 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.975205 15810 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:35.918179 15881 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:35.932492 15953 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:35.979336 15810 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.976874 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:35.986593 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:35.987283 16051 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:35.987777 16051 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:35.987891 15941 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:35.988651 16051 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.989343 15989 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:35.989765 15989 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:35.990881 16141 log.cc:927] Time spent T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:16:35.990535 15989 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:35.991670 15989 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:35.993690 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:35.995301 15987 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:35.996372 15987 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:35.996681 15987 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:35.997375 15987 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.998167 16082 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:35.998441 15987 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:35.999719 15940 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:36.027186 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 857937fa61dc400a81345944f0a14511 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:16:35.973770 15817 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:35.989766 15881 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:35.981191 15882 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:35.995172 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.012385 11365 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:35.988153 16105 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:36.003906 16096 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:36.028993 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.973398 15885 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:36.044974 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:36.046685 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:35.976348 15936 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:16:35.973954 16068 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:36.072845 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 77734.3us
I20260709 13:16:35.931375 16104 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.089583 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:36.092936 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:36.068933 15882 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.025962 15549 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:36.063599 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.090698 11368 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:36.104000 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:36.070690 16077 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.111897 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:36.084079 16082 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:36.072625 16105 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:36.092339 16098 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:36.122866 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.075997 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.097477 16068 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.070313 15940 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.128077 11517 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:16:36.129446 11513 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:36.130786 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:36.105463 15634 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:36.105789 15852 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:36.109261 15882 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.108048 16076 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:36.115908 11366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:36.118521 16082 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.094291 15885 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.105005 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:36.137792 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:36.220361 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.305375 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:36.305692 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 77519.4us
I20260709 13:16:36.306035 11512 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.130311 16077 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:36.160953 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.326537 11512 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:36.328096 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.219712 11361 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:16:36.307188 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 87012.3us
I20260709 13:16:36.158913 11367 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:16:36.191541 12034 leader_election.cc:400] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
W20260709 13:16:36.181916 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 763740us
W20260709 13:16:36.332538 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 141195us
W20260709 13:16:36.304548 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 166832us
I20260709 13:16:36.191752 15861 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 13:16:36.189785 16037 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 13:16:36.187995 16092 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:36.310809 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 234886us
I20260709 13:16:36.311317 11439 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:36.341221 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:36.311930 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.312260 16107 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:36.310678 16051 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:36.345058 12032 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:36.319629 15850 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:36.335284 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 230431us
W20260709 13:16:36.175971 16144 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:16:36.336768 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:36.326479 11366 raft_consensus.cc:606] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:36.150043 16098 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:36.139672 16068 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.187433 15698 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.195080 15865 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.375917 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:36.330729 16082 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:36.192852 15901 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.378924 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.193018 16054 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:36.324332 16096 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:36.192672 15989 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:36.347316 15606 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:36.304682 15958 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.213179 15881 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.330591 15852 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:36.306702 16012 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:16:36.343360 16036 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16.
I20260709 13:16:36.393875 15817 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:36.399928 16142 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.374s	user 0.003s	sys 0.000s
I20260709 13:16:36.162058 15549 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.234s
I20260709 13:16:36.189091 15903 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:36.400435 11262 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:36.369962 16034 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 13:16:36.423362 11511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.080s	sys 0.153s
W20260709 13:16:36.453733 11511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.080s	sys 0.153s
I20260709 13:16:36.423238 16051 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:36.440196 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 95508.4us
I20260709 13:16:36.347782 16092 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.406581 15852 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.386380 15953 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:36.400993 15606 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:36.445827 16101 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:36.446102 15928 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:36.446784 15817 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:36.457476 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 75985.5us
I20260709 13:16:36.388399 15430 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:16:36.328019 16105 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.453090 15740 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:36.388288 16054 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.372517 16107 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.449731 11366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a38969de5c824718aa81b9622624bc51"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:36.386535 15865 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.391754 16096 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.384780 15698 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:36.486094 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 141102us
I20260709 13:16:36.460520 15810 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:36.402287 15989 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.388062 15901 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.490972 16167 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:36.492887 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.472466 15850 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:36.473102 15606 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:36.480759 16101 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:36.487221 16104 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.461907 15903 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:36.488075 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 141023us
I20260709 13:16:36.507977 16037 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:36.520063 16037 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:36.520908 11262 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:36.522794 16092 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:36.524748 15865 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.525326 16107 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.525646 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.525801 16106 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:36.509004 11263 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39444 (request call id 353) took 8237 ms. Trace:
I20260709 13:16:36.526850 15740 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:36.527765 15740 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:36.510715 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:36.529970 16075 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:36.516109 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.522665 11262 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 8 to 9, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:36.513367 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.519661 15698 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.532567 11262 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 4 to 7, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:36.534541 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.526264 11511 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:16:36.538266 16152 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 13:16:36.539377 16153 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:16:36.551889 15810 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.553818 15901 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.526894 11263 rpcz_store.cc:276] 0709 13:16:28.271735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.271907 (+   172us) service_pool.cc:224] Handling call
0709 13:16:36.508794 (+8236887us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:29.066374 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:29.091256 (+ 24882us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:29.091291 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:16:29.091303 (+    12us) tablet.cc:662] Decoding operations
0709 13:16:29.283968 (+192665us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:29.287769 (+  3801us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:29.290462 (+  2693us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:16:29.312194 (+ 21732us) tablet.cc:701] Row locks acquired
0709 13:16:29.312218 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:16:29.312379 (+   161us) write_op.cc:270] Start()
0709 13:16:29.312518 (+   139us) write_op.cc:276] Timestamp: P: 1783602989312327 usec, L: 0
0709 13:16:29.312545 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:16:29.313278 (+   733us) log.cc:844] Serialized 5453 byte log entry
0709 13:16:29.783293 (+470015us) op_driver.cc:464] REPLICATION: finished
0709 13:16:30.146485 (+363192us) write_op.cc:301] APPLY: starting
0709 13:16:30.151200 (+  4715us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:30.313598 (+162398us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:30.313723 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:30.395058 (+ 81335us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:30.415501 (+ 20443us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 13:16:30.417607 (+  2106us) write_op.cc:312] APPLY: finished
0709 13:16:30.425359 (+  7752us) log.cc:844] Serialized 129 byte log entry
0709 13:16:30.425779 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:30.426281 (+   502us) write_op.cc:454] Released schema lock
0709 13:16:30.426385 (+   104us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:30.427164 (+   779us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:30.427200 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35731,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":304246,"apply.run_cpu_time_us":24965,"apply.run_wall_time_us":288971,"cfile_cache_hit":54,"cfile_cache_hit_bytes":56376,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6215,"cfile_init":2,"lbm_read_time_us":28780,"lbm_reads_1-10_ms":3,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":8,"num_ops":14,"prepare.queue_time_us":162229,"prepare.run_cpu_time_us":21025,"prepare.run_wall_time_us":348996,"raft.queue_time_us":381995,"raft.run_cpu_time_us":13066,"raft.run_wall_time_us":169450,"replication_time_us":470602,"spinlock_wait_cycles":65920,"thread_start_us":662340,"threads_started":4,"wal-append.queue_time_us":85440,"wal-append.run_cpu_time_us":37174,"wal-append.run_wall_time_us":2029883}]]}
I20260709 13:16:36.551688 16161 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:36.537539 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:36.555318 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.496954 15953 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:16:36.493404 16134 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:16:36.534375 16174 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.525015 16051 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.556694 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:36.591838 11262 catalog_manager.cc:5697] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:36.511672 16173 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.595933 11409 leader_election.cc:400] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12.
I20260709 13:16:36.517418 15970 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:36.612462 11361 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:16:36.648988 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:36.586131 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68109.7us
I20260709 13:16:36.517637 15850 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.619926 11262 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 3 to 5, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 5 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:36.656845 11517 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:16:36.575874 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.658701 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.570310 16096 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.660058 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.660691 12068 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:36.659498 12065 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:36.661067 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:36.662942 12063 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:16:36.640846 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 45028.4us
I20260709 13:16:36.650663 11262 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 9 to 12, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 12 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:36.661310 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.652669 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:36.665593 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.664767 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:36.653620 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:36.668910 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:36.560184 16107 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:36.611302 16150 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:16:36.683543 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.684234 12064 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:36.685262 11511 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:36.567624 15940 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.561076 15958 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.670686 11443 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:36.572671 15903 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.664198 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:36.647394 15970 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.673367 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:36.583536 16151 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:16:36.712837 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:36.647532 15987 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.000s
W20260709 13:16:36.610489 16148 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:16:36.568583 15885 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.680931 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.571413 15989 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.670596 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:36.564798 16068 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:36.689007 16054 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.690013 15882 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.727571 15882 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.728240 12064 raft_consensus.cc:1217] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Deduplicated request from leader. Original: 9.10->[10.11-10.11]   Dedup: 10.11->[]
I20260709 13:16:36.728765 12064 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:16:36.729111 11369 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:36.690503 15850 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.578922 15606 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.730784 11441 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:36.732278 15606 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.585850 16105 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:36.691941 16034 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:36.586290 16092 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:16:36.735775 15698 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:36.736156 15698 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:36.574080 16174 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:36.722157 15953 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:36.738173 16174 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:36.695617 12034 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:36.585762 16101 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:36.739100 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 43565.4us
I20260709 13:16:36.740240 16101 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.740850 16034 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:36.741318 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:36.742059 15634 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:36.742681 15861 consensus_queue.cc:1059] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:36.743381 15861 consensus_queue.cc:1243] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:36.742980 15698 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.744184 15698 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:36.744702 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:36.745582 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.746336 11510 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:36.695643 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.697499 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:36.747728 11439 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
W20260709 13:16:36.647429 16156 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:16:36.653857 16036 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:16:36.747174 16054 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:16:36.653966 16173 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:36.580476 16147 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:16:36.672642 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:36.750366 11517 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:36.741966 16034 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:16:36.728823 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.751472 16173 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:36.576480 16082 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.712929 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:36.759572 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 46710us
I20260709 13:16:36.760262 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:36.568601 16104 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:36.610023 16164 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:36.660760 15865 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.718526 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.766736 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.719971 11364 raft_consensus.cc:2410] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:36.714584 15970 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.722571 15810 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:16:36.725489 16068 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:36.747043 15989 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:36.753746 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.747145 15940 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:16:36.755141 12062 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:16:36.756058 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:36.781090 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 250832us
I20260709 13:16:36.789578 11437 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:36.790520 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.787422 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:36.778326 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 441648us
I20260709 13:16:36.794889 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:36.796267 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.791158 11437 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:36.757673 11515 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:16:36.762887 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.763563 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.763841 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.764204 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:36.762328 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:36.764997 11517 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:36.801257 11441 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:36.802062 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:36.762590 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.571584 16167 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:36.767706 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 306522us
I20260709 13:16:36.768776 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.806046 11365 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:16:36.702952 11516 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:36.775672 12063 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:36.809754 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:36.816200 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:36.817137 11441 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:36.817319 11368 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:36.765669 16036 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:36.816875 12032 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:36.817924 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.818557 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.822132 11517 raft_consensus.cc:2393] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:36.768301 15861 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:36.826445 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:36.766103 16101 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:36.766004 15953 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:36.761471 16104 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:36.824343 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.730657 15817 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:36.825073 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.827246 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.862422 11436 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 7.
I20260709 13:16:36.863027 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:36.787441 16098 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:36.832523 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.869256 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2410) took 1061 ms. Trace:
I20260709 13:16:36.831184 12062 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:36.778007 15865 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:36.825855 15861 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:36.785931 15989 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:36.863492 16036 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:36.820436 16158 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:36.857934 16164 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:36.885597 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.811342 15760 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.859282 11441 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:36.834405 15430 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:36.860057 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:36.860563 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44435.1us
W20260709 13:16:36.887826 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 286205us
I20260709 13:16:36.888100 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:36.824759 16173 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:16:36.852618 16174 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:36.825706 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.865476 11511 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:36.866382 11484 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.859966 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.869938 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:36.916944 11445 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:36.897392 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.864732 16104 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:36.833703 15936 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:16:36.869695 11519 rpcz_store.cc:276] 0709 13:16:35.816293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.816509 (+   216us) service_pool.cc:224] Handling call
0709 13:16:35.826923 (+ 10414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:35.827226 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.827253 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:35.833848 (+  6595us) log.cc:844] Serialized 26 byte log entry
0709 13:16:35.979020 (+145172us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:35.979062 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.979131 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:36.579929 (+600798us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:36.858907 (+278978us) raft_consensus.cc:1685] finished
0709 13:16:36.858930 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.869185 (+ 10255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":447,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4180,"spinlock_wait_cycles":37828096,"thread_start_us":126295,"threads_started":1,"wal-append.queue_time_us":281236}
I20260709 13:16:36.769531 16169 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:16:36.882525 11335 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.933046 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.872911 16160 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:16:36.886456 11436 raft_consensus.cc:2393] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:36.890763 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:36.941211 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:36.915891 16107 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:36.892401 11515 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:36.897682 11510 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:16:36.923025 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56719.2us
I20260709 13:16:36.922463 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.917765 16164 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:36.927023 15926 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:36.947783 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.923902 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:36.924412 11517 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:16:36.924641 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:36.863687 15885 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:36.877544 15861 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:36.834225 16129 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:16:36.936874 12034 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:37.000772 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 76939.2us
W20260709 13:16:37.001317 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 64550.2us
I20260709 13:16:37.001518 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:36.937716 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:36.997571 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.888233 16163 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:36.894944 15430 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:36.939878 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 57469.5us
I20260709 13:16:36.942426 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.891300 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.929210 16077 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:36.858204 15953 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:36.919970 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:37.010015 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.857568 12064 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:36.959064 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:37.011168 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:36.962064 11516 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:16:36.963317 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.962883 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:36.918243 15760 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.973515 11510 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: [8.8-11.9]
I20260709 13:16:36.968811 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.928951 16096 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:36.967352 15882 raft_consensus.cc:993] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:36.998402 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.936116 15989 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:36.965426 15941 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:37.008240 11436 raft_consensus.cc:2410] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:16:37.008282 11366 raft_consensus.cc:606] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:37.008530 11511 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:37.015753 11517 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:37.013827 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:36.943943 16168 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:16:37.033259 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:37.035388 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 231283us
I20260709 13:16:37.043442 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:37.049620 11442 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:37.007566 15606 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:37.009825 15850 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:37.007174 15430 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:37.036497 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.011307 15909 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.054818 11436 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:37.057809 11440 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:36.968672 15926 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.009584 16077 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:37.073124 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.027231 16096 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:37.029918 15698 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:37.031703 16173 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:37.060029 15936 raft_consensus.cc:1081] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
W20260709 13:16:36.998765 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 176086us
I20260709 13:16:37.177644 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:37.079847 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:37.059783 15850 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.018096 16163 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.064453 16169 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:37.071461 15941 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:37.072623 16129 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:37.029670 16054 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:16:36.982487 16098 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.149008 12033 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:37.093115 15989 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:37.151177 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7a5787304dfe4697baeaeb367ee7dda3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 01ae50c877d04ba687e4e06259ba5581 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:37.152469 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 99497.2us
I20260709 13:16:37.088542 15903 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:16:37.059749 16106 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
W20260709 13:16:37.155197 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 153781us
I20260709 13:16:37.285435 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.054009 16076 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:36.969395 16036 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:37.148392 15852 consensus_meta.cc:287] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.662s	user 0.013s	sys 0.001s
I20260709 13:16:37.170667 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2478) took 1125 ms. Trace:
I20260709 13:16:37.140547 15953 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:37.174507 11445 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:37.180265 11408 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:37.278155 15865 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:37.385623 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 205416us
I20260709 13:16:37.386536 11408 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:37.188400 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 115317us
I20260709 13:16:37.061399 16037 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:37.190905 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 125352us
I20260709 13:16:37.156082 15817 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:37.169787 16096 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:37.169971 16077 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:37.281286 11253 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 130151us
I20260709 13:16:37.293758 15882 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.178882 16173 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.295239 16076 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:37.193348 16169 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:37.190320 15850 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:37.295938 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 216413us
I20260709 13:16:37.298738 11369 rpcz_store.cc:276] 0709 13:16:36.045342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.045487 (+   145us) service_pool.cc:224] Handling call
0709 13:16:36.045822 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:36.729515 (+683693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.170458 (+440943us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54000a6ed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:37.170616 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1571035904}
W20260709 13:16:37.407336 16189 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 13:16:37.195843 16112 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:37.072297 16164 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:37.373807 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 88636.8us
W20260709 13:16:37.404820 16188 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
W20260709 13:16:37.406322 15970 consensus_meta.cc:287] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.002s
W20260709 13:16:37.281843 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 84276.7us
W20260709 13:16:37.419054 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 456209us
W20260709 13:16:37.461159 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 69423.7us
W20260709 13:16:37.463743 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 77308.7us
I20260709 13:16:37.294745 16165 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:37.170173 15698 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:37.481300 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.483563 16173 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.383038 15634 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:37.485898 16177 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.477s	user 0.011s	sys 0.000s
W20260709 13:16:37.408161 16183 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:16:37.500320 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 351387us
I20260709 13:16:37.527436 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:37.529886 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.409840 16158 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:16:37.539145 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.539237 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.489313 16076 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:37.373800 15852 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:37.409935 16168 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:37.560356 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.560791 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:37.533165 16174 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.560248 15760 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.552397 15953 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:37.484503 15882 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.563825 16160 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:37.584636 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.561997 16158 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:37.327528 15941 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:37.587041 11365 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:16:37.604434 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.578030 15634 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:37.606216 15634 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:37.578112 15903 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.607821 15903 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.609880 15903 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:37.610464 15882 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.583206 16169 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.588793 15817 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:37.612169 16186 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:37.611340 16169 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:37.613143 16196 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:16:37.613922 16192 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:37.614276 15926 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.614786 16192 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:37.615152 16192 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:37.615902 16192 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:37.617103 16192 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:37.611385 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 1961) took 1451 ms. Trace:
I20260709 13:16:37.619882 11362 rpcz_store.cc:276] 0709 13:16:36.160305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.160546 (+   241us) service_pool.cc:224] Handling call
0709 13:16:37.611149 (+1450603us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54002dcb54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:37.611328 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357939456}
I20260709 13:16:37.628242 15606 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:37.628810 15606 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.636066 16193 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:37.636840 16193 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:37.637801 16193 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.639552 16207 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:37.641441 16191 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
W20260709 13:16:37.642522 16178 consensus_queue.cc:1175] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6e87b213a4554d94825aceec352fc094" responder_term: 9 status { last_received { term: 8 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:37.639391 16185 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.159s
W20260709 13:16:37.644271 16200 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:16:37.645550 15989 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:37.646737 16012 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:37.637678 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50722 (request call id 33) took 1188 ms. Trace:
I20260709 13:16:37.647274 11366 rpcz_store.cc:276] 0709 13:16:36.449413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.449579 (+   166us) service_pool.cc:224] Handling call
0709 13:16:37.637447 (+1187868us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54002dc654. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f8853 00007f35c19c660a 00007f35baf4c7d4 00007f35baf4c762 00007f35baf4c691 00007f35baf4c60c 00007f35baf4b8c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:37.637568 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182590848}
I20260709 13:16:37.550613 15970 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.649039 11443 raft_consensus.cc:2393] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:37.577270 16165 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
W20260709 13:16:37.650434 15909 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.001s
I20260709 13:16:37.650866 15909 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:37.651986 16012 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:37.652472 16208 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:37.652823 16012 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.653128 15606 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.653993 16012 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.655134 15606 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.655651 11366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
W20260709 13:16:37.656005 16198 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 13:16:37.656628 16182 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.484s	user 0.001s	sys 0.000s
I20260709 13:16:37.657030 16180 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:37.662181 16180 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.665010 16180 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.657441 16171 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:37.666003 16171 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.668027 16171 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.668644 11366 raft_consensus.cc:606] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:37.657716 16076 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.652189 15970 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.659017 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:37.671320 11445 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:37.671921 16076 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:37.672660 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.673102 15958 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:37.674247 15958 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:37.675809 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:37.673060 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:37.680022 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:37.683451 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 79086.4us
W20260709 13:16:37.683909 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 399211us
W20260709 13:16:37.678284 16190 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.460s	user 0.001s	sys 0.001s
I20260709 13:16:37.653015 16208 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:37.615854 15852 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:37.705744 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:37.663254 15901 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:37.652185 16163 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.705210 11518 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:37.719462 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:37.716761 11513 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:16:37.691998 16076 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.692006 16192 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:37.719233 12066 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:37.694883 15549 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:37.702123 16169 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.673142 16180 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:16:37.583441 15936 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.124s
W20260709 13:16:37.708333 16189 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:16:37.723963 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:37.727221 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:37.728947 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:37.587530 16174 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:37.728938 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:37.701997 16207 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.759994 16215 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:16:37.760926 15903 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:37.761740 16202 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:16:37.762701 16204 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:37.717582 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:37.763230 16204 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.764954 11513 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:37.765384 16204 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.707125 15953 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:37.768148 15953 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.770583 15953 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:37.702724 16101 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:16:37.719676 15901 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:37.759881 16187 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.555s	user 0.000s	sys 0.001s
I20260709 13:16:37.730453 15549 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:37.774235 15549 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.775517 15549 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:37.776129 16174 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:37.731803 16076 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.759820 16207 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.778380 16076 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:37.720817 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 58079.9us
I20260709 13:16:37.783702 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:37.786518 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:37.701902 15970 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:16:37.773902 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2648) took 1106 ms. Trace:
I20260709 13:16:37.787977 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:37.779848 15430 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.000s
I20260709 13:16:37.799714 15430 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.801990 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:37.780114 15852 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:37.804226 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 119537us
I20260709 13:16:37.779443 16207 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:37.822850 11409 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 2.033s (SENT)
I20260709 13:16:37.788378 12069 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:37.812008 15926 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.816787 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1314) took 1157 ms. Trace:
W20260709 13:16:37.821718 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 260048us
I20260709 13:16:37.796123 11438 rpcz_store.cc:276] 0709 13:16:36.667879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.668659 (+   780us) service_pool.cc:224] Handling call
0709 13:16:36.695948 (+ 27289us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:36.696291 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:36.696318 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:36.696840 (+   522us) log.cc:844] Serialized 52 byte log entry
0709 13:16:36.876976 (+180136us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:36.877014 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.877095 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:37.773666 (+896571us) raft_consensus.cc:1685] finished
0709 13:16:37.773691 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:37.773835 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1850,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":9984,"thread_start_us":179768,"threads_started":1,"wal-append.queue_time_us":313736}
I20260709 13:16:37.824167 12068 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:37.821388 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:37.825091 12061 rpcz_store.cc:276] 0709 13:16:36.659061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.659168 (+   107us) service_pool.cc:224] Handling call
0709 13:16:36.682923 (+ 23755us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:36.720665 (+ 37742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:36.720697 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:36.731107 (+ 10410us) log.cc:844] Serialized 52 byte log entry
0709 13:16:36.944694 (+213587us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:36.965454 (+ 20760us) log.cc:844] Serialized 14 byte log entry
0709 13:16:36.965666 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.965730 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:37.606880 (+641150us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b48001b0e58. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f 00007f35c1a15f37 00007f35c1a2a55c 00007f35c1a2a508 00007f35c1a2a488 00007f35c1a2a443 00007f35c1a296ec 00007f35c0953a96
0709 13:16:37.805515 (+198635us) raft_consensus.cc:1685] finished
0709 13:16:37.816535 (+ 11020us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:37.816715 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47998,"prepare.queue_time_us":25923,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":39016,"raft.run_cpu_time_us":10742,"raft.run_wall_time_us":641808,"spinlock_wait_cycles":3993984,"thread_start_us":213196,"threads_started":1,"wal-append.queue_time_us":295633}
I20260709 13:16:37.830009 11334 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.753357 16169 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:37.841543 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:37.824626 15987 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.810084 16167 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:37.810891 15852 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:37.877415 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:37.883605 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 53694.1us
I20260709 13:16:37.878345 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:37.879113 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 963) took 1044 ms. Trace:
I20260709 13:16:37.878100 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:37.929574 11441 rpcz_store.cc:276] 0709 13:16:36.835080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.011981 (+176901us) service_pool.cc:224] Handling call
0709 13:16:37.048585 (+ 36604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.062702 (+ 14117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.062735 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.072957 (+ 10222us) log.cc:844] Serialized 26 byte log entry
0709 13:16:37.338215 (+265258us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:37.845410 (+507195us) log.cc:844] Serialized 14 byte log entry
0709 13:16:37.878785 (+ 33375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.878867 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:37.878910 (+    43us) raft_consensus.cc:1685] finished
0709 13:16:37.878921 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:37.879045 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46852,"prepare.queue_time_us":239638,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":66375,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9865,"thread_start_us":516432,"threads_started":2,"wal-append.queue_time_us":321332}
I20260709 13:16:37.809911 16192 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:16:37.894140 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.731263 16158 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:37.934283 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 56943.9us
I20260709 13:16:37.942093 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:37.831349 16174 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:37.787838 16214 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:16:37.877494 16051 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:37.927416 15861 consensus_meta.cc:287] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.871s	user 0.007s	sys 0.003s
I20260709 13:16:37.827529 15926 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.955899 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:37.965698 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:37.885957 15953 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:37.887375 15909 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:16:37.886390 16180 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:37.930572 16167 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.940788 16207 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:16:38.025677 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.837970 16173 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.927744 15987 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:37.952903 16186 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:37.953933 16213 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:38.073148 11444 raft_consensus.cc:1217] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Deduplicated request from leader. Original: 7.7->[8.8-11.9]   Dedup: 8.8->[11.9-11.9]
W20260709 13:16:38.012270 16209 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:16:37.997228 16206 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:38.009210 15634 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:38.033010 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:38.011623 16158 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:38.145179 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:38.072811 16160 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:38.092604 16104 consensus_meta.cc:287] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.158s	user 0.010s	sys 0.010s
I20260709 13:16:38.072470 15989 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:38.149948 11517 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:16:37.998337 16051 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:37.833568 16208 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.121217 16180 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.082376 16167 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:38.140573 16214 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:37.998745 15861 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.158174 16104 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.164688 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:37.949725 16174 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:38.167558 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 237318us
I20260709 13:16:38.174849 11444 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: [8.8-11.9]
I20260709 13:16:38.155519 15989 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:38.213161 11512 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:38.217363 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:38.222803 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:38.033021 15852 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.153985 15634 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.230970 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:38.231640 12067 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:38.232506 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:38.233062 12067 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:38.233806 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:38.234887 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:38.235500 12062 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:38.237435 11439 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:16:38.238812 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.157636 16160 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.242362 11365 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:16:38.242276 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:38.243865 11369 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:16:38.245404 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.246110 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:38.228849 16218 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.400s	user 0.000s	sys 0.001s
I20260709 13:16:38.223486 16098 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:38.249658 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:38.240880 15989 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.255963 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:38.256217 16206 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:38.256724 16222 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:16:38.259148 16096 consensus_meta.cc:287] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.772s	user 0.006s	sys 0.000s
W20260709 13:16:38.261101 16220 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 13:16:38.237413 16166 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:38.237915 15430 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.326696 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:38.265152 16174 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.225126 16211 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:38.273413 16175 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.416s	user 0.003s	sys 0.010s
W20260709 13:16:38.274372 16224 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:16:38.275900 16173 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.277222 16051 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:38.277778 16077 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.789s	user 0.008s	sys 0.000s
I20260709 13:16:38.279865 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.349455 11445 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:38.317857 11365 raft_consensus.cc:2393] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 13: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:38.349848 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.286134 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 970) took 1350 ms. Trace:
W20260709 13:16:38.246158 16225 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:16:38.335319 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:38.208645 16214 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.228345 16036 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:16:38.327890 16096 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.336367 15989 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:38.362778 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:38.343842 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 310859us
I20260709 13:16:38.350492 11436 rpcz_store.cc:276] 0709 13:16:36.935627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.577912 (+642285us) service_pool.cc:224] Handling call
0709 13:16:37.578277 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.583881 (+  5604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.583917 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.584387 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:16:37.948250 (+363863us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:37.948305 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.948372 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:38.285917 (+337545us) raft_consensus.cc:1685] finished
0709 13:16:38.285938 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:38.286074 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5392,"raft.queue_time_us":18408,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":94975,"thread_start_us":360889,"threads_started":1,"wal-append.queue_time_us":571813}
I20260709 13:16:38.329121 16098 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.355935 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:38.353631 15850 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.001s
I20260709 13:16:38.370085 16211 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:38.370579 16211 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.370915 16211 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.340098 15926 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.372815 11445 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:16:38.356222 11519 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
I20260709 13:16:38.382714 16169 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.383796 16104 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.373566 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.364490 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:38.340428 16213 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:38.391722 11515 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:38.388984 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:38.394779 11334 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.656s (SENT) [suppressed 17 similar messages]
I20260709 13:16:38.374643 15987 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:38.409220 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.399121 16211 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.417232 15634 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:38.356267 16077 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.394364 15926 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:38.332712 16206 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.433542 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:38.400647 16051 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:38.434218 11510 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:38.434283 11436 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:16:38.436726 11436 pending_rounds.cc:85] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 6
I20260709 13:16:38.437021 11436 pending_rounds.cc:107] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:16:38.437240 11436 raft_consensus.cc:2889] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:38.438244 11516 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:38.377902 15850 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.440263 11510 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:38.377213 16098 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.439080 11519 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:38.442047 11519 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:38.394073 16173 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:16:38.426752 16012 consensus_meta.cc:287] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.770s	user 0.008s	sys 0.004s
I20260709 13:16:38.444072 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 968) took 1537 ms. Trace:
I20260709 13:16:38.444903 11510 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:38.448633 12060 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:38.433195 15926 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:38.452561 16233 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:16:38.436491 16051 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.447271 11440 rpcz_store.cc:276] 0709 13:16:36.906914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.173736 (+266822us) service_pool.cc:224] Handling call
0709 13:16:37.200015 (+ 26279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.543077 (+343062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.543113 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.555826 (+ 12713us) log.cc:844] Serialized 26 byte log entry
0709 13:16:37.801744 (+245918us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:37.843496 (+ 41752us) log.cc:844] Serialized 14 byte log entry
0709 13:16:37.844048 (+   552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.844124 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:38.443882 (+599758us) raft_consensus.cc:1685] finished
0709 13:16:38.443900 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:38.444013 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4082,"prepare.queue_time_us":74433,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4427,"raft.queue_time_us":244941,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":14580,"spinlock_wait_cycles":31675136,"thread_start_us":451802,"threads_started":2,"wal-append.queue_time_us":471396}
I20260709 13:16:38.439692 16180 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.425032 16245 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:16:38.446902 15861 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:38.468221 11519 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.453223 15926 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.458639 16051 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:38.460175 16211 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.487043 16160 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:38.490711 16230 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:16:38.491057 16231 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:16:38.450575 15606 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:16:38.491365 16214 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.452802 16213 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:38.485212 15549 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.709s	user 0.007s	sys 0.002s
W20260709 13:16:38.500085 16232 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.333s	user 0.000s	sys 0.009s
I20260709 13:16:38.502769 11516 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:16:38.504347 16241 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:38.512369 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.514300 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:38.513182 16096 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.490248 16245 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:38.513846 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.516867 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.524091 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.496410 16160 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:38.514295 16236 log.cc:927] Time spent T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:16:38.530673 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.533768 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:38.536228 16082 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.515621 16206 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.537361 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.538194 11363 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:38.539366 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2485) took 1746 ms. Trace:
I20260709 13:16:38.512226 16241 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:38.540429 11370 rpcz_store.cc:276] 0709 13:16:36.793179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.796031 (+  2852us) service_pool.cc:224] Handling call
0709 13:16:38.539187 (+1743156us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001e50d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:38.539312 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2955673984}
I20260709 13:16:38.547556 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.517083 16246 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:38.540300 16158 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:38.552836 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.554853 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:38.561748 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:38.554212 16158 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.565349 16206 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:38.566252 16251 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:38.566812 16251 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.534740 16160 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.568857 16251 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:38.569188 16160 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:38.570078 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.570667 11436 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:38.571748 16206 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:38.573177 16207 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:38.573670 16207 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.576150 16098 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.576850 16077 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.528527 16214 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:38.532578 16248 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:38.661080 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:38.661686 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.662911 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:38.663543 12063 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:38.664108 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1029) took 2009 ms. Trace:
I20260709 13:16:38.664462 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:38.664857 16186 consensus_meta.cc:287] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.525s	user 0.004s	sys 0.004s
I20260709 13:16:38.665287 16186 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:38.664465 11512 rpcz_store.cc:276] 0709 13:16:36.654955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.656215 (+  1260us) service_pool.cc:224] Handling call
0709 13:16:36.656501 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.217361 (+1560860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.663906 (+446545us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54000659d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:38.664053 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3579638656}
I20260709 13:16:38.665074 12062 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:38.669114 15926 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.672396 15926 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.683058 15850 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.684373 16241 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.685786 16241 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:38.688987 11441 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:38.583284 16207 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.696888 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.701990 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:38.702618 11436 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.708570 16158 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.642418 16214 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.572182 16167 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.713538 16167 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:38.714799 12061 raft_consensus.cc:2368] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
I20260709 13:16:38.526527 16245 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.717468 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:38.698951 16178 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:38.721151 11441 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:38.730446 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.732522 16176 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:16:38.754304 16092 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:38.754760 16092 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:38.755682 16092 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.756705 16092 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:38.751127 11367 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:38.632967 16246 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.740926 16214 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.773111 11436 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
W20260709 13:16:38.776198 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 381417us
I20260709 13:16:38.776830 11335 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:38.729461 16178 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.716094 16248 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.785748 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:38.801952 16037 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:38.803737 16248 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.804298 16248 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:16:38.803113 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:38.805555 16165 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:38.806105 16178 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.808338 11334 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:38.809180 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:38.809841 15881 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:38.810205 15881 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.810524 15881 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.810762 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 971) took 1155 ms. Trace:
I20260709 13:16:38.811610 16248 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:38.783449 15817 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:38.812054 16248 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.812407 16248 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.785748 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:38.785161 16245 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:38.787006 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:38.816679 12063 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.797231 16246 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.799849 11510 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:38.813248 12064 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:38.811185 11439 rpcz_store.cc:276] 0709 13:16:37.655837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.657398 (+  1561us) service_pool.cc:224] Handling call
0709 13:16:37.657737 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.240426 (+582689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.810459 (+570033us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54001b6cd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:38.810693 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1330560896}
I20260709 13:16:38.826407 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.818215 15817 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:38.829171 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:38.833441 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2432) took 1013 ms. Trace:
I20260709 13:16:38.833820 11517 rpcz_store.cc:276] 0709 13:16:37.820176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.820335 (+   159us) service_pool.cc:224] Handling call
0709 13:16:37.825006 (+  4671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.156065 (+331059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.833204 (+677139us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b5400109354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:38.833382 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746732800}
I20260709 13:16:38.835291 15817 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.837075 15852 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:38.840997 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:38.842211 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:38.844746 11519 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.845288 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:36660 (request call id 2437) took 1023 ms. Trace:
I20260709 13:16:38.845582 11519 rpcz_store.cc:276] 0709 13:16:37.828470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.356087 (+527617us) service_pool.cc:224] Handling call
0709 13:16:38.843661 (+487574us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b54000654d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f5a3d 00007f35c19c5cd0 00007f35baf4a254 00007f35baf4a1e2 00007f35baf4a111 00007f35baf4a08c 00007f35baf49342 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:38.845185 (+  1524us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135489408}
I20260709 13:16:38.849917 11437 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
W20260709 13:16:38.849855 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2510) took 2106 ms (client timeout 1534 ms). Trace:
W20260709 13:16:38.850800 11443 rpcz_store.cc:269] 0709 13:16:36.743487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.743633 (+   146us) service_pool.cc:224] Handling call
0709 13:16:38.849625 (+2105992us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b5400289bd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:38.849775 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078092032}
I20260709 13:16:38.851037 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:38.852252 11438 raft_consensus.cc:2393] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:38.852628 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2520) took 1147 ms. Trace:
I20260709 13:16:38.852900 11438 rpcz_store.cc:276] 0709 13:16:37.705103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.885660 (+180557us) service_pool.cc:224] Handling call
0709 13:16:38.850888 (+965228us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800078098. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:16:38.852578 (+  1690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28362624}
I20260709 13:16:38.857062 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 974) took 1200 ms. Trace:
I20260709 13:16:38.857451 11444 rpcz_store.cc:276] 0709 13:16:37.656649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.660334 (+  3685us) service_pool.cc:224] Handling call
0709 13:16:37.660614 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:38.214930 (+554316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.856795 (+641865us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54002099d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:38.857002 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948285312}
I20260709 13:16:38.858616 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:38.872309 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1135) took 2000 ms. Trace:
I20260709 13:16:38.893333 11367 rpcz_store.cc:276] 0709 13:16:36.871757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.010418 (+138661us) service_pool.cc:224] Handling call
0709 13:16:37.612097 (+601679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.750943 (+1138846us) spinlock_profiling.cc:243] Waited 66.7 ms on lock 0x7b54004d1aa0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc975dc0 00007f35bca001c0 00007f35bca01231 00007f35bc9fdd8f 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c
0709 13:16:38.762351 (+ 11408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.872101 (+109750us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54002dcb50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:38.872244 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536507904}
I20260709 13:16:38.878003 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1327) took 1084 ms. Trace:
I20260709 13:16:38.894860 12068 rpcz_store.cc:276] 0709 13:16:37.793446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.796243 (+  2797us) service_pool.cc:224] Handling call
0709 13:16:37.823974 (+ 27731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:37.826156 (+  2182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.877774 (+1051618us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b540034b550. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:38.877933 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62975360}
I20260709 13:16:38.896253 12063 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:38.863343 15901 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.897769 15901 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:38.898232 15698 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.899183 15698 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:38.884013 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1041) took 1121 ms. Trace:
I20260709 13:16:38.906431 11511 rpcz_store.cc:276] 0709 13:16:37.762840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.766928 (+  4088us) service_pool.cc:224] Handling call
0709 13:16:37.767278 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.790215 (+ 22937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.790252 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.790805 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.121602 (+330797us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:38.121711 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.865033 (+743322us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b48001b0cd8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f 00007f35c1a15f37 00007f35c1a2a55c 00007f35c1a2a508 00007f35c1a2a488 00007f35c1a2a443 00007f35c1a296ec 00007f35c0953a96
0709 13:16:38.883685 (+ 18652us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5400371254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:38.883744 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:38.883789 (+    45us) raft_consensus.cc:1685] finished
0709 13:16:38.883801 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:38.883939 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12431,"raft.queue_time_us":13395,"raft.run_cpu_time_us":4421,"raft.run_wall_time_us":1068452,"spinlock_wait_cycles":31548928,"thread_start_us":330400,"threads_started":1,"wal-append.queue_time_us":547875}
I20260709 13:16:38.885915 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2442) took 1041 ms. Trace:
I20260709 13:16:38.908581 11514 rpcz_store.cc:276] 0709 13:16:37.844240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.497293 (+653053us) service_pool.cc:224] Handling call
0709 13:16:38.516116 (+ 18823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.575912 (+ 59796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.576070 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:38.576120 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:38.576153 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.885689 (+309536us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b540031e554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:38.885737 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:38.885858 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136952704}
W20260709 13:16:38.912263 11253 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica b1906ab37ba541f493936685a808c8a9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:38.914142 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.914081 16106 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.914778 16106 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.916858 16106 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:38.917163 11409 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Timed out: RequestConsensusVote RPC to 127.10.4.65:42857 timed out after 2.414s (SENT)
I20260709 13:16:38.917703 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:38.919754 16193 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:38.921882 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:38.922760 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.952275 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1044) took 1108 ms. Trace:
I20260709 13:16:38.952701 11510 rpcz_store.cc:276] 0709 13:16:37.843999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.447850 (+603851us) service_pool.cc:224] Handling call
0709 13:16:38.448120 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.819276 (+371156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.952043 (+132767us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54002a76d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:38.952216 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808371072}
I20260709 13:16:38.953459 16076 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:38.954064 16076 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.956496 16076 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:38.859314 11444 raft_consensus.cc:2393] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:38.927806 16106 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:38.929248 16214 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:38.957866 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 957) took 2143 ms. Trace:
I20260709 13:16:38.958256 16214 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.931067 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 886) took 1053 ms. Trace:
I20260709 13:16:38.929529 16193 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.970054 16193 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.971024 15953 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.971560 15953 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:38.972460 16193 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.967550 16129 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:38.974076 16129 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.973340 12060 rpcz_store.cc:276] 0709 13:16:37.877559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.877829 (+   270us) service_pool.cc:224] Handling call
0709 13:16:38.930893 (+1053064us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54004d8d54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:38.931004 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1193290496}
I20260709 13:16:38.952316 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:38.960424 16214 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.969950 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:38.980424 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:38.981546 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.982501 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.982681 16214 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:38.983814 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.984933 15606 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:38.973423 15953 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:38.986369 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:38.985183 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:38.933980 15936 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:38.937281 16036 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:38.992561 16036 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.994640 16036 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:38.938464 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.997331 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:38.997061 15606 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.998363 15852 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:38.998409 16174 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:38.999010 16174 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.000443 16250 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.000941 16250 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.002373 15606 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.003440 15430 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:39.001068 16174 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.005003 15606 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.995234 15936 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.942384 16221 raft_consensus.cc:993] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:39.006922 16221 raft_consensus.cc:1081] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:39.007030 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.009203 15936 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:38.940759 15987 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:39.016216 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:39.017030 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.017392 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.018186 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.019313 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.020821 11442 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:38.982518 16129 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:39.025112 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.027076 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.027786 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.028394 11367 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:39.028683 11367 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:39.029584 11519 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:39.030824 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.031543 11514 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:39.032327 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:39.033804 12033 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:39.034581 11511 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:39.000283 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:38.817862 16051 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:38.958221 11437 rpcz_store.cc:276] 0709 13:16:36.814369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.814506 (+   137us) service_pool.cc:224] Handling call
0709 13:16:38.849743 (+2035237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.850420 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.957702 (+107282us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400289bd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:38.957814 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4679998848}
I20260709 13:16:38.998955 15852 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.006201 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.052917 12064 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
W20260709 13:16:38.914834 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:39.053766 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:39.054189 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.055326 11437 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:16:39.038273 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.892897 12065 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:39.044013 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.059257 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:38.944244 11334 leader_election.cc:400] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
I20260709 13:16:39.061237 11334 leader_election.cc:403] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:16:39.018498 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.027834 11510 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:16:39.063359 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 141550us
I20260709 13:16:39.064621 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.067663 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.068397 11511 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
W20260709 13:16:39.072564 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 72343us
I20260709 13:16:39.029390 11367 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.073390 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.073484 11367 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:39.076898 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:39.078127 15698 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:39.078374 15852 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.071473 15953 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:39.057252 16162 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.079607 15698 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:39.079888 16169 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:39.002974 16250 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:39.080478 15698 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.066888 16217 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:39.081570 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1328) took 1273 ms. Trace:
I20260709 13:16:39.081790 15430 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:16:39.067267 16214 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:39.083123 16082 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.083896 16203 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.051549 16129 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.067463 15987 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.052078 16106 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:39.097860 15760 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:39.098002 15987 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:39.098460 15760 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.099067 15936 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:39.098922 16162 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.099640 15936 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.102202 11368 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:16:39.089785 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:39.103363 11442 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:39.090778 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.105831 11441 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:39.114151 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.124415 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.124456 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.125965 12063 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:39.082114 12066 rpcz_store.cc:276] 0709 13:16:37.807809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.808059 (+   250us) service_pool.cc:224] Handling call
0709 13:16:37.808399 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:37.808778 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.808805 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:37.812595 (+  3790us) log.cc:844] Serialized 52 byte log entry
0709 13:16:38.844283 (+1031688us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b18002bfcc0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:39.027235 (+182952us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:39.027280 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.027346 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.051565 (+ 24219us) raft_consensus.cc:1685] finished
0709 13:16:39.051592 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.081496 (+ 29904us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1370,"raft.queue_time_us":2434,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8908,"spinlock_wait_cycles":15111424,"thread_start_us":182737,"threads_started":1,"wal-append.queue_time_us":1214576}
W20260709 13:16:39.135955 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 191822us
I20260709 13:16:39.137403 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:39.138974 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.140060 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.141323 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:39.142393 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 69088.6us
I20260709 13:16:39.142714 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:39.143471 11438 raft_consensus.cc:2368] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12.
I20260709 13:16:39.143761 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.144920 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.146708 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.147220 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.151973 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.153513 12060 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:39.156127 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.159907 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.161501 11514 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:16:39.162181 11408 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:39.100998 15430 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:39.168797 16211 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:39.169904 16213 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:39.170147 16077 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:39.139104 16221 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:39.172672 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:39.173125 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.173560 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.178403 11442 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:39.163441 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:39.183553 16246 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:39.184026 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.184602 15606 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:39.185060 15850 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:39.185570 16082 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.186822 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.141650 16193 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.224838 16213 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:39.217420 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54052.8us
I20260709 13:16:39.217741 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:39.229328 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 55865.7us
I20260709 13:16:39.218273 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.123948 15698 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:39.236029 16174 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.236884 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:39.237098 16193 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:39.189728 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.161051 16075 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:39.163590 15852 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:39.213912 11334 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.228760 16213 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:39.244328 11440 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.059s	sys 0.072s
I20260709 13:16:39.244920 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.081598 16217 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.227303 16241 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:39.252416 12067 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
W20260709 13:16:39.252871 11440 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.060s	sys 0.072s
I20260709 13:16:39.171447 15881 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.283365 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:39.284204 12067 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:39.180709 16051 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:39.291904 12065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1149) took 2517 ms (client timeout 1656 ms). Trace:
I20260709 13:16:39.268184 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.270327 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.304536 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:39.275247 11365 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.010s	sys 0.019s
I20260709 13:16:39.179278 16163 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:16:39.171965 15430 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:39.151345 11367 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
W20260709 13:16:39.279431 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 61731.5us
I20260709 13:16:39.213078 16186 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:16:39.254441 16213 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:39.319805 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.266381 12060 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:39.228039 16248 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:39.307512 11365 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.011s	sys 0.019s
I20260709 13:16:39.274073 15850 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:39.311723 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:39.343302 12067 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:39.305176 12066 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:39.273841 16214 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:39.238543 16082 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:39.235718 16280 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:16:39.307915 12065 rpcz_store.cc:269] 0709 13:16:36.763784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:36.763983 (+   199us) service_pool.cc:224] Handling call
0709 13:16:39.281275 (+2517292us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400741dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:39.281546 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4796446464}
I20260709 13:16:39.349885 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.318710 11370 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:39.315145 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:39.293627 16215 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:39.329483 16129 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.368046 11362 raft_consensus.cc:2368] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12.
I20260709 13:16:39.335099 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.344064 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:39.314095 12062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.015s	sys 0.019s
W20260709 13:16:39.369282 12062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.015s	sys 0.020s
I20260709 13:16:39.327188 12034 leader_election.cc:400] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12.
I20260709 13:16:39.313104 15760 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.354620 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.368937 12060 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 8: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:16:39.346009 11436 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.023s	sys 0.018s
W20260709 13:16:39.370643 11436 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.023s	sys 0.018s
I20260709 13:16:39.371071 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.371637 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.372138 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.372912 12060 raft_consensus.cc:2410] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:16:39.370028 12034 leader_election.cc:403] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:16:39.358995 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:39.376417 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68820.2us
I20260709 13:16:39.376524 16155 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12.
I20260709 13:16:39.378168 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:39.378913 16106 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:39.379472 16106 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:39.328732 16077 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:39.336184 16134 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.329623 16250 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:39.311775 16208 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:16:39.381732 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.354568 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.387354 11361 raft_consensus.cc:2410] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:16:39.388602 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.389276 12060 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:39.390107 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:39.345983 16205 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:39.391390 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.398635 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.404261 11483 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.406769 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.408793 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.385159 15850 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:39.411051 16241 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:39.412463 16076 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:39.413703 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.414436 16076 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:39.415683 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:39.416406 16076 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:39.416780 16076 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.417112 16076 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:39.417203 15958 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:39.417763 15958 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:39.401464 16107 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.000s	sys 0.016s
I20260709 13:16:39.387145 15987 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:39.431704 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.397073 15885 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:39.455533 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:39.456565 12066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:39.380375 16106 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.400933 16155 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:39.397779 16215 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:39.352311 16214 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:39.455164 11485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.012s	sys 0.014s
I20260709 13:16:39.407428 16245 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:39.459623 11485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.013s	sys 0.014s
I20260709 13:16:39.442492 16076 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.383368 16129 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.456902 16205 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:16:39.391623 16134 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:39.454993 16107 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.001s	sys 0.016s
I20260709 13:16:39.426832 16169 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:39.467103 15817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.013s	sys 0.012s
I20260709 13:16:39.489904 16248 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.384367 15430 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.399562 16250 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.457113 16241 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:39.468796 16075 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:39.468340 16155 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.485255 15940 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.513353 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.486403 16174 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:39.510818 11519 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:39.511808 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:39.519744 11512 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:39.520128 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.521572 11519 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:16:39.512080 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.513115 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.523744 11513 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:39.513620 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.523233 11515 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:16:39.494233 15817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.013s	sys 0.012s
I20260709 13:16:39.518277 15928 consensus_queue.cc:1059] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:16:39.533488 16221 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
I20260709 13:16:39.536914 16076 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.525411 15940 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.496011 16169 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.539379 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.540310 11515 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:39.540480 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.541030 11518 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:39.541306 16169 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.539431 15940 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:39.503996 16241 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:39.543996 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.544677 11518 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:39.545571 16134 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:39.547700 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.552731 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.554574 11516 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:39.555022 11368 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:39.556495 16217 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:39.560475 16268 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:16:39.563994 16082 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.565707 16129 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:39.566147 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.567062 11512 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:39.568511 16214 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.569391 16215 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.569957 11512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
I20260709 13:16:39.570637 11512 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:39.571019 11512 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:39.571131 16082 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.571719 16082 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.464402 15958 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.575433 16082 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.498530 16075 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.576930 16075 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:39.578511 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.515558 15881 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:39.593555 11512 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:39.597362 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.598160 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.598924 11366 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:39.599080 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.598196 11367 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:39.602916 12067 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:39.614691 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.616289 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.621061 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:39.622766 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.623588 11515 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:39.624521 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.625108 11515 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:39.638000 11512 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:39.614542 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:39.656289 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:39.657001 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.657142 12067 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:39.658900 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:39.647846 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:39.663131 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:39.658051 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.668340 11515 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:39.673087 16214 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:39.684332 11333 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.674714 15881 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:39.678427 16155 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:39.678188 16075 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.667090 16205 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:39.680510 15903 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:39.680841 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:39.674736 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2497) took 1119 ms. Trace:
I20260709 13:16:39.683439 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:39.691684 12067 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:39.692550 12062 raft_consensus.cc:2410] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:39.693414 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:39.694211 11514 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:39.683746 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:39.658082 16217 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:39.591853 16107 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:39.544021 16241 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:39.674595 16178 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.684711 16214 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:39.699321 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:39.691148 11363 rpcz_store.cc:276] 0709 13:16:38.555487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.555696 (+   209us) service_pool.cc:224] Handling call
0709 13:16:38.563729 (+  8033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.803639 (+239910us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:38.989763 (+186124us) log.cc:844] Serialized 14 byte log entry
0709 13:16:39.346976 (+357213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:39.347039 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:39.347088 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.674480 (+327392us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54000dbe54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:39.674537 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.674674 (+   137us) inbound_call.cc:177] Queueing success response
0709 13:16:39.676384 (+  1710us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b4800002958. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f 00007f35c1a15f37 00007f35c1a2a55c 00007f35c1a2a508 00007f35c1a2a488 00007f35c1a2a443 00007f35c1a296ec 00007f35c0953a96
Metrics: {"mutex_wait_us":21093,"prepare.queue_time_us":185580,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":31571,"raft.run_cpu_time_us":5758,"raft.run_wall_time_us":309439,"spinlock_wait_cycles":555296384,"thread_start_us":515674,"threads_started":2,"wal-append.queue_time_us":501804}
I20260709 13:16:39.704733 11515 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:39.705271 11519 raft_consensus.cc:2368] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:16:39.708632 12069 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:16:39.689570 16155 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:39.688630 16197 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:39.688021 16219 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:16:39.694398 16205 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:39.730074 11445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.023s	sys 0.044s
W20260709 13:16:39.690662 15901 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.016s	sys 0.020s
I20260709 13:16:39.696251 16217 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:39.722144 16203 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:39.729869 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:39.733610 11445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.023s	sys 0.045s
I20260709 13:16:39.717633 12067 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:39.715196 16197 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:39.711117 11333 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:39.737107 11511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.030s	sys 0.008s
I20260709 13:16:39.710291 16155 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:39.749545 11511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.031s	sys 0.008s
W20260709 13:16:39.746903 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 533031us
W20260709 13:16:39.750689 12064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.010s	sys 0.023s
W20260709 13:16:39.752012 12064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.010s	sys 0.023s
I20260709 13:16:39.748371 11333 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:39.751573 12062 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:16:39.739956 15901 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.016s	sys 0.020s
W20260709 13:16:39.740648 15953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.017s	sys 0.024s
I20260709 13:16:39.738462 11515 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:39.745901 16217 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:39.750782 15970 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:39.745914 16203 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.750519 15881 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:39.795195 11440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.006s	sys 0.033s
W20260709 13:16:39.795744 11440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.007s	sys 0.033s
I20260709 13:16:39.751332 16155 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:39.806816 11514 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:39.787717 12062 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:39.796700 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2669) took 1272 ms. Trace:
I20260709 13:16:39.815276 11440 rpcz_store.cc:276] 0709 13:16:38.524236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.524348 (+   112us) service_pool.cc:224] Handling call
0709 13:16:38.539144 (+ 14796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.539495 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:39.278011 (+738516us) log.cc:844] Serialized 14 byte log entry
0709 13:16:39.796259 (+518248us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:39.796315 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:39.796365 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.796504 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.796646 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26237,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":15263,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":5306,"thread_start_us":1256001,"threads_started":2}
I20260709 13:16:39.756397 16221 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:39.814388 15698 consensus_meta.cc:287] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.578s	user 0.001s	sys 0.009s
W20260709 13:16:39.812809 15953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.017s	sys 0.027s
W20260709 13:16:39.825079 11408 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.018s	sys 0.025s
I20260709 13:16:39.796377 16203 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:39.813657 16192 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:39.760311 15817 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:16:39.829276 11515 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
W20260709 13:16:39.876863 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 75689.4us
I20260709 13:16:39.877383 11515 pending_rounds.cc:85] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 6
I20260709 13:16:39.877741 11515 pending_rounds.cc:107] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 12.7
W20260709 13:16:39.851344 11408 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.018s	sys 0.026s
I20260709 13:16:39.878034 11515 raft_consensus.cc:2889] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:39.852427 12060 raft_consensus.cc:2410] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:16:39.760464 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.020s	sys 0.017s
W20260709 13:16:39.878894 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.020s	sys 0.019s
W20260709 13:16:39.879163 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 652196us
I20260709 13:16:39.878389 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:39.749332 16214 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:39.878300 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 596444us
I20260709 13:16:39.887979 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.888427 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.888695 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.890157 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.891242 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.892220 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.893180 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.893697 11364 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:39.887877 11367 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:16:39.895555 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.897002 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.896109 11367 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:16:39.900784 11515 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:39.902891 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.903704 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.904343 11370 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:39.904302 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:39.902809 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:39.906148 11366 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:39.893961 11364 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:39.908833 11364 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:39.912243 11364 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:39.915851 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.917225 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:39.919014 11367 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
W20260709 13:16:39.919602 11410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.013s	sys 0.028s
W20260709 13:16:39.919915 11410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.013s	sys 0.028s
W20260709 13:16:39.921154 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.923631 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.924379 11444 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:39.884632 11365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.030s
W20260709 13:16:39.925366 11365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.030s
I20260709 13:16:39.926122 11364 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:39.929692 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:39.930266 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:39.934055 11515 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
W20260709 13:16:39.938232 11510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.016s	sys 0.018s
W20260709 13:16:39.978644 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 49022.4us
I20260709 13:16:39.935930 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:39.975234 11510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.018s	sys 0.020s
I20260709 13:16:39.979796 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:39.977056 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.980600 11368 raft_consensus.cc:2376] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15. Re-sending same reply.
I20260709 13:16:39.982358 11441 raft_consensus.cc:2410] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:39.987031 11370 raft_consensus.cc:2410] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:16:39.990625 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.991321 11441 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:39.993891 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.996743 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:39.995195 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.008s	sys 0.021s
W20260709 13:16:39.997413 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.008s	sys 0.021s
W20260709 13:16:39.997942 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 626343us
I20260709 13:16:39.930603 15850 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:40.000844 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:40.001452 11363 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:40.002166 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:39.935601 16076 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:39.821954 16205 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:39.977614 16281 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:16:40.007645 11370 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:16:39.909989 15881 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:39.928622 16278 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:16:40.011457 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:40.014863 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:40.015339 16292 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:16:39.937640 15882 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:39.814947 16197 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:39.990434 16245 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:40.017776 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:39.928946 16214 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:40.006973 15953 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:40.024437 16279 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
I20260709 13:16:40.027294 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2512) took 2344 ms. Trace:
I20260709 13:16:39.975299 16163 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:16:40.032299 11436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.006s	sys 0.025s
W20260709 13:16:40.345968 11436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.006s	sys 0.025s
I20260709 13:16:40.346709 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2531) took 1650 ms. Trace:
I20260709 13:16:39.936877 16075 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.038735 16203 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:40.016788 16082 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:40.017714 16129 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:40.050305 16169 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:40.096040 16192 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:40.112720 16287 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
W20260709 13:16:40.056134 16193 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.020s	sys 0.027s
W20260709 13:16:40.193785 12035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.015s	sys 0.035s
W20260709 13:16:40.188057 15958 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.009s	sys 0.010s
W20260709 13:16:40.198621 16215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.013s	sys 0.007s
W20260709 13:16:40.189910 16266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.026s	sys 0.033s
W20260709 13:16:40.198153 16106 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.007s	sys 0.013s
W20260709 13:16:40.198800 15928 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.018s	sys 0.035s
W20260709 13:16:40.238059 11518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.006s	sys 0.022s
W20260709 13:16:40.554363 11518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.008s	sys 0.024s
W20260709 13:16:40.197459 11361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.011s	sys 0.011s
W20260709 13:16:40.555678 11361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.015s	sys 0.012s
W20260709 13:16:40.154490 11513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.019s	sys 0.007s
W20260709 13:16:40.556105 11513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.024s	sys 0.008s
W20260709 13:16:40.201503 11512 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.018s	sys 0.005s
I20260709 13:16:40.059484 16297 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:40.282572 12063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.019s	sys 0.017s
W20260709 13:16:40.556955 12063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.020s	sys 0.018s
I20260709 13:16:40.282531 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2500) took 1302 ms. Trace:
I20260709 13:16:40.557605 11362 rpcz_store.cc:276] 0709 13:16:38.980072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.980178 (+   106us) service_pool.cc:224] Handling call
0709 13:16:40.099496 (+1119318us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54001e55d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:40.282465 (+182969us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889706752}
I20260709 13:16:40.321867 11443 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:16:40.316277 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 314146us
W20260709 13:16:40.556761 11512 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.020s	sys 0.008s
I20260709 13:16:40.560391 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:40.305642 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.502664 15958 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.010s	sys 0.010s
I20260709 13:16:40.342336 15882 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:40.315939 16213 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.970s	user 0.011s	sys 0.025s
W20260709 13:16:40.320425 16288 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.265s	user 0.007s	sys 0.008s
I20260709 13:16:40.345047 11445 rpcz_store.cc:276] 0709 13:16:37.682541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.378364 (+695823us) service_pool.cc:224] Handling call
0709 13:16:38.378622 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.378936 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.378962 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:38.848483 (+469521us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b2c00060428. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:38.848887 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:16:39.746706 (+897819us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:39.746752 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.746831 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.026925 (+280094us) raft_consensus.cc:1685] finished
0709 13:16:40.026951 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.027223 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":535,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":19587456,"thread_start_us":897464,"threads_started":1,"wal-append.queue_time_us":1170381}
I20260709 13:16:40.347096 11436 rpcz_store.cc:276] 0709 13:16:38.696279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.776033 (+ 79754us) service_pool.cc:224] Handling call
0709 13:16:38.785665 (+  9632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.785895 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:38.787154 (+  1259us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.346350 (+1559196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.346405 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:40.346457 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.346526 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.346653 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11702,"prepare.queue_time_us":1112,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":127658,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":13180,"thread_start_us":1546747,"threads_started":2,"wal-append.queue_time_us":851721}
W20260709 13:16:40.354743 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 343348us
W20260709 13:16:40.502772 12035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.019s	sys 0.035s
W20260709 13:16:40.504657 11483 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.003s	sys 0.029s
W20260709 13:16:40.581748 11483 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.004s	sys 0.029s
W20260709 13:16:40.581987 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.582072 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 602328us
W20260709 13:16:40.544006 12062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.064s	sys 0.146s
I20260709 13:16:40.351293 15936 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.588555 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:40.589447 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.018s	sys 0.028s
W20260709 13:16:40.548719 11367 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.026s	sys 0.043s
I20260709 13:16:40.549180 11369 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:40.342574 12067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.003s	sys 0.041s
W20260709 13:16:40.591226 12067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.003s	sys 0.041s
I20260709 13:16:40.564332 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:40.504657 16193 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.022s	sys 0.030s
I20260709 13:16:40.541764 16082 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:40.570277 11365 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.017s	sys 0.054s
W20260709 13:16:40.544996 16215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.013s	sys 0.008s
I20260709 13:16:40.358222 16207 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:40.548425 16106 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.008s	sys 0.014s
I20260709 13:16:40.502226 16245 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:40.552151 16266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.026s	sys 0.034s
I20260709 13:16:40.341619 15881 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.491986 16192 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:40.553853 15928 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.018s	sys 0.036s
W20260709 13:16:40.584152 12062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.064s	sys 0.146s
I20260709 13:16:40.506599 15885 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:40.555148 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2450) took 1006 ms. Trace:
I20260709 13:16:40.606758 16193 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:40.591835 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.018s	sys 0.028s
W20260709 13:16:40.628050 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 980258us
I20260709 13:16:40.592073 12068 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:40.630141 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:40.631367 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1329) took 1576 ms. Trace:
W20260709 13:16:40.620436 16316 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:16:40.592474 11367 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.027s	sys 0.043s
W20260709 13:16:40.593528 11442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.013s	sys 0.033s
W20260709 13:16:40.502044 15901 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.014s	sys 0.032s
I20260709 13:16:40.589612 12061 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
W20260709 13:16:40.601511 11366 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.021s	sys 0.026s
W20260709 13:16:40.637403 11366 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.022s	sys 0.026s
I20260709 13:16:40.589797 16213 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:40.608083 11411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.019s	sys 0.029s
W20260709 13:16:40.644455 11411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.020s	sys 0.029s
I20260709 13:16:40.648975 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:40.357940 15953 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.590095 16301 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:40.599900 16287 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:16:40.506367 16169 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.589964 16302 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:40.620594 12066 raft_consensus.cc:606] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:40.594585 16186 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:40.623648 11518 rpcz_store.cc:276] 0709 13:16:39.549101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.550806 (+  1705us) service_pool.cc:224] Handling call
0709 13:16:39.551102 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:39.551299 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:39.551642 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.554800 (+1003158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.554857 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:40.554905 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.554972 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.555092 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29728,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":15936,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":14140,"thread_start_us":1002799,"threads_started":1}
I20260709 13:16:40.563794 12065 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 14. Ops: []
W20260709 13:16:40.625491 16299 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.305s	user 0.000s	sys 0.000s
W20260709 13:16:40.625886 16107 consensus_meta.cc:287] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.918s	user 0.014s	sys 0.020s
I20260709 13:16:40.570290 16303 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:16:40.629019 12032 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:40.674715 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45778.7us
I20260709 13:16:40.635838 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:40.631893 12064 rpcz_store.cc:276] 0709 13:16:39.055373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.058600 (+  3227us) service_pool.cc:224] Handling call
0709 13:16:39.058943 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.068364 (+  9421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.068397 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.075128 (+  6731us) log.cc:844] Serialized 26 byte log entry
0709 13:16:39.752590 (+677462us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:39.752635 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.752697 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.631176 (+878479us) raft_consensus.cc:1685] finished
0709 13:16:40.631195 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.631312 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9421,"raft.queue_time_us":9735,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":29104,"thread_start_us":677058,"threads_started":1,"wal-append.queue_time_us":956965}
W20260709 13:16:40.491592 16293 consensus_queue.cc:1175] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b1906ab37ba541f493936685a808c8a9" responder_term: 11 status { last_received { term: 11 index: 7 } last_committed_idx: 5 last_received_current_leader { term: 11 index: 7 } }
I20260709 13:16:40.575078 16297 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:40.626636 16300 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:16:40.659063 15882 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.681380 15881 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.623553 15885 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:40.682605 15885 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:40.683173 16298 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:40.683708 16298 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.666154 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:40.676292 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:40.686902 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1885) took 1657 ms. Trace:
W20260709 13:16:40.636765 15901 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.015s	sys 0.032s
I20260709 13:16:40.632840 16193 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:40.651026 16036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.032s	sys 0.031s
I20260709 13:16:40.669453 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:40.693214 12065 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:40.638394 16219 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:40.675962 12068 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:40.671726 16186 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.675359 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:40.677471 16301 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.659273 15928 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:40.677727 16310 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:16:40.677417 16303 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:40.693020 16036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.032s	sys 0.031s
I20260709 13:16:40.675384 16107 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:16:40.692888 16217 consensus_meta.cc:287] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.888s	user 0.019s	sys 0.013s
I20260709 13:16:40.693228 16193 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:40.715009 16301 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.715878 16298 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.711321 16303 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:40.717244 16301 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.717723 16303 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:40.718928 16303 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:40.719301 16217 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.711881 15928 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.721482 16217 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:40.669114 16082 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:40.724805 16155 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.900s	user 0.010s	sys 0.000s
I20260709 13:16:40.725822 16169 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.726439 16306 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:40.673633 16302 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.708652 16203 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:40.676708 16245 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.729233 16306 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.704176 16219 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:40.753105 16306 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:40.773445 16250 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:40.804081 16318 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:16:40.811685 16302 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.822932 16306 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:40.616259 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.002s	sys 0.049s
W20260709 13:16:40.618211 11365 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.017s	sys 0.055s
W20260709 13:16:40.999277 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.002s	sys 0.049s
W20260709 13:16:40.999774 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 970873us
I20260709 13:16:41.000830 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2498) took 2439 ms. Trace:
I20260709 13:16:41.001330 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:40.643211 11438 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.013s	sys 0.065s
W20260709 13:16:41.005376 11438 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.013s	sys 0.065s
W20260709 13:16:40.633605 11442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.013s	sys 0.033s
I20260709 13:16:41.007887 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2548) took 1064 ms. Trace:
I20260709 13:16:41.008296 11442 rpcz_store.cc:276] 0709 13:16:39.943518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.943675 (+   157us) service_pool.cc:224] Handling call
0709 13:16:39.944046 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.944378 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.944402 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.976688 (+ 32286us) log.cc:844] Serialized 26 byte log entry
0709 13:16:41.007241 (+1030553us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:41.007314 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.007428 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:41.007491 (+    63us) raft_consensus.cc:1685] finished
0709 13:16:41.007500 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.007711 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26969,"raft.queue_time_us":399015,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":157718,"thread_start_us":987370,"threads_started":1,"wal-append.queue_time_us":759245}
I20260709 13:16:41.010183 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.010962 11442 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:40.641901 11443 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:40.687258 11517 rpcz_store.cc:276] 0709 13:16:39.029168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.029275 (+   107us) service_pool.cc:224] Handling call
0709 13:16:39.032524 (+  3249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.032901 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:39.033270 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 13:16:39.459585 (+426315us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.474661 (+ 15076us) log.cc:844] Serialized 26 byte log entry
0709 13:16:39.474798 (+   137us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:39.474831 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.474897 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.686719 (+1211822us) raft_consensus.cc:1685] finished
0709 13:16:40.686738 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.686850 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48424,"prepare.queue_time_us":184,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":19547,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":23107,"thread_start_us":425798,"threads_started":1,"wal-append.queue_time_us":725526}
I20260709 13:16:40.695705 12064 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:40.707106 12065 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:41.013342 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2550) took 1066 ms. Trace:
I20260709 13:16:40.871928 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:40.873919 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.013890 11438 rpcz_store.cc:276] 0709 13:16:39.947076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.948570 (+  1494us) service_pool.cc:224] Handling call
0709 13:16:39.993288 (+ 44718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:41.006030 (+1012742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:41.006095 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:41.006134 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:41.006166 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.006236 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.013267 (+  7031us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11516,"raft.queue_time_us":363555,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":55403,"thread_start_us":1012172,"threads_started":1}
I20260709 13:16:41.015341 11436 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:40.874930 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.016814 11444 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
W20260709 13:16:40.876468 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 300090us
I20260709 13:16:40.877521 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1054) took 1254 ms. Trace:
I20260709 13:16:41.019233 11512 rpcz_store.cc:276] 0709 13:16:39.622652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.638996 (+ 16344us) service_pool.cc:224] Handling call
0709 13:16:39.639318 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:39.639591 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:40.560149 (+920558us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.876551 (+316402us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.876667 (+   116us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:40.876810 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.876922 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.877221 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32092,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":60107,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":287,"thread_start_us":1235833,"threads_started":2,"wal-append.queue_time_us":230311}
I20260709 13:16:40.878785 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1353) took 1289 ms. Trace:
W20260709 13:16:40.879681 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 298051us
I20260709 13:16:41.001291 11365 rpcz_store.cc:276] 0709 13:16:38.560850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.563929 (+  3079us) service_pool.cc:224] Handling call
0709 13:16:38.564305 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.564510 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:39.345324 (+780814us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.999854 (+1654530us) log.cc:844] Serialized 14 byte log entry
0709 13:16:41.000194 (+   340us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:41.000269 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:41.000381 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.000488 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.000651 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64555,"prepare.queue_time_us":503958,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":51292,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":8556,"thread_start_us":2275948,"threads_started":3,"wal-append.queue_time_us":795645}
I20260709 13:16:40.881696 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 892) took 1548 ms. Trace:
I20260709 13:16:40.882341 11439 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:16:40.887684 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1347) took 1573 ms. Trace:
W20260709 13:16:40.897858 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.938632 11367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:34564
I20260709 13:16:40.943783 16037 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:40.725220 16155 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:40.946067 15928 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:40.955947 16186 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:40.968429 16336 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.969452 16303 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:40.975596 16337 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.021399 12062 rpcz_store.cc:276] 0709 13:16:39.588754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.878532 (+289778us) service_pool.cc:224] Handling call
0709 13:16:39.913266 (+ 34734us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.913596 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.913623 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.914060 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:16:40.621785 (+707725us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:40.621825 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.621888 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.878084 (+256196us) raft_consensus.cc:1685] finished
0709 13:16:40.878128 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.878444 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15502,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":44646,"thread_start_us":707410,"threads_started":1,"wal-append.queue_time_us":749472}
I20260709 13:16:41.023958 16337 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.023504 16321 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.977384 16302 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:40.982445 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:41.027117 16321 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.027608 16245 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.028239 11437 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:40.982955 16331 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.028514 16245 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.029466 16331 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.030514 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.004539 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:40.686015 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1890) took 1405 ms. Trace:
I20260709 13:16:41.028990 16037 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.035576 16037 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:41.037324 15852 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.038499 16155 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.038157 15852 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.012023 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2689) took 1007 ms. Trace:
I20260709 13:16:41.022167 12067 rpcz_store.cc:276] 0709 13:16:39.333547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.747541 (+413994us) service_pool.cc:224] Handling call
0709 13:16:39.747970 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.748359 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.748388 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.753007 (+  4619us) log.cc:844] Serialized 26 byte log entry
0709 13:16:40.591634 (+838627us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:40.591673 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.591738 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.881198 (+289460us) raft_consensus.cc:1685] finished
0709 13:16:40.881229 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.881484 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1133,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":1253,"thread_start_us":838321,"threads_started":1,"wal-append.queue_time_us":889692}
I20260709 13:16:41.027956 16302 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.043996 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.022536 16068 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:41.032001 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.039636 12063 rpcz_store.cc:276] 0709 13:16:39.314207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.698100 (+383893us) service_pool.cc:224] Handling call
0709 13:16:39.698503 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.698905 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.698934 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.703457 (+  4523us) log.cc:844] Serialized 26 byte log entry
0709 13:16:40.557323 (+853866us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:40.557360 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.557422 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.887234 (+329812us) raft_consensus.cc:1685] finished
0709 13:16:40.887266 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.887537 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4705,"thread_start_us":826112,"threads_started":1,"wal-append.queue_time_us":1021084}
I20260709 13:16:41.033104 15958 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:41.047773 11334 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:41.041131 11443 rpcz_store.cc:276] 0709 13:16:40.004862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.005051 (+   189us) service_pool.cc:224] Handling call
0709 13:16:41.011781 (+1006730us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:41.045118 11518 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:41.049391 16336 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.041131 16337 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.051930 12061 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:41.006767 16214 consensus_meta.cc:287] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.655s	user 0.005s	sys 0.007s
I20260709 13:16:41.021518 16306 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.077474 15926 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.077760 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.052116 16192 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.039628 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.077492 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:41.077540 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 58274.3us
I20260709 13:16:41.076823 16068 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:41.082887 11515 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:16:41.084637 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.084900 11518 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:41.085301 16331 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:41.082227 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 81105us
I20260709 13:16:41.088426 11510 rpcz_store.cc:276] 0709 13:16:39.280995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.281409 (+   414us) service_pool.cc:224] Handling call
0709 13:16:39.281765 (+   356us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:39.307749 (+ 25984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.307782 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:16:39.308436 (+   654us) log.cc:844] Serialized 78 byte log entry
0709 13:16:39.981427 (+672991us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:39.981468 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.986048 (+  4580us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.685773 (+699725us) raft_consensus.cc:1685] finished
0709 13:16:40.685793 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.685953 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25533,"raft.queue_time_us":25499,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":10838,"thread_start_us":632152,"threads_started":1,"wal-append.queue_time_us":1002420}
I20260709 13:16:41.086386 16068 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.090252 16068 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.090526 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.091475 11511 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:41.048839 16303 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.078433 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.047660 15958 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.098142 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:41.092448 16204 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.099813 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.102427 11514 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:41.106761 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.106781 16082 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.106709 15936 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.107857 16331 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.105360 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.102752 16214 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:41.108666 15936 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.114817 15852 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.109982 15882 raft_consensus.cc:2764] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:16:41.115885 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.116281 16162 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.116578 11510 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:41.117738 15936 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.119213 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2509) took 1228 ms. Trace:
I20260709 13:16:41.119642 11364 rpcz_store.cc:276] 0709 13:16:39.890793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.927708 (+ 36915us) service_pool.cc:224] Handling call
0709 13:16:39.927990 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.860974 (+932984us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:40.861802 (+   828us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.946673 (+ 84871us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.946733 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:40.947904 (+  1171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.118790 (+170886us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b540035c354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:41.118847 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.119032 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30194,"prepare.queue_time_us":75852,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":68,"raft.queue_time_us":57581,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":187,"spinlock_wait_cycles":2143657728,"thread_start_us":50115,"threads_started":1,"wal-append.queue_time_us":91497}
I20260709 13:16:41.121619 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.122932 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.125833 16303 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.126945 16306 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:41.108417 16082 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.114322 16192 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.105149 15928 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:41.130096 15852 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.130620 15928 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.130769 15852 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.131892 16302 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.132805 16344 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:41.133270 16344 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.133672 16344 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.133270 16321 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:41.135913 16312 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.468s	user 0.009s	sys 0.012s
W20260709 13:16:41.138741 16311 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.466s	user 0.006s	sys 0.039s
I20260709 13:16:41.126863 16297 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.140174 11514 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
W20260709 13:16:41.140213 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 119154us
I20260709 13:16:41.141206 16174 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:41.141482 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.141861 16174 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:41.145038 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 1.48999e+06us
I20260709 13:16:41.145802 16297 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:41.146888 11442 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:41.132930 15928 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.108753 16331 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.149271 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48186 (request call id 6) took 1818 ms. Trace:
I20260709 13:16:41.103364 16204 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.149915 16331 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.147317 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:41.149708 12066 rpcz_store.cc:276] 0709 13:16:39.330783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.427442 (+ 96659us) service_pool.cc:224] Handling call
0709 13:16:41.148910 (+1721468us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54005014d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f8853 00007f35c19c660a 00007f35baf4c7d4 00007f35baf4c762 00007f35baf4c691 00007f35baf4c60c 00007f35baf4b8c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:41.149034 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2675661696}
I20260709 13:16:41.123286 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2459) took 1545 ms. Trace:
W20260709 13:16:41.153676 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 428222us
I20260709 13:16:41.111666 16336 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.154901 11519 rpcz_store.cc:276] 0709 13:16:39.578110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.578227 (+   117us) service_pool.cc:224] Handling call
0709 13:16:41.122873 (+1544646us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54002a76d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:41.123098 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289770752}
I20260709 13:16:41.160843 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.162751 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 888) took 1830 ms. Trace:
I20260709 13:16:41.162993 16162 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.167483 12060 rpcz_store.cc:276] 0709 13:16:39.332564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.389898 (+ 57334us) service_pool.cc:224] Handling call
0709 13:16:41.162368 (+1772470us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540052b7d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:41.162590 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061277440}
I20260709 13:16:41.128801 16306 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:41.162617 16346 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:16:41.171842 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.173291 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2460) took 1594 ms. Trace:
I20260709 13:16:41.171041 16306 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.174676 11513 rpcz_store.cc:276] 0709 13:16:39.578414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.579168 (+   754us) service_pool.cc:224] Handling call
0709 13:16:39.579537 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:39.579908 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.579937 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:39.580537 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 13:16:40.556457 (+975920us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:40.585739 (+ 29282us) log.cc:844] Serialized 14 byte log entry
0709 13:16:40.873720 (+287981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.873925 (+   205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:41.137849 (+263924us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:41.169734 (+ 31885us) raft_consensus.cc:1685] finished
0709 13:16:41.171929 (+  2195us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.173136 (+  1207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11262,"prepare.queue_time_us":41920,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":37779,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":62471,"spinlock_wait_cycles":11545344,"thread_start_us":1251616,"threads_started":2,"wal-append.queue_time_us":1035803}
I20260709 13:16:41.177855 16336 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:41.182402 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 134877us
I20260709 13:16:41.185195 11368 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.185985 16208 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.194196 15958 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.201417 16205 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:41.202127 16205 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.203336 16197 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.204690 16205 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.208447 16193 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.205113 16208 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.186097 16336 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.194069 16037 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.215107 16208 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.221819 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.222637 16037 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.219457 16068 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.208246 16344 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.232185 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.233911 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.234577 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.235561 11513 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:41.237968 16207 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.238782 16207 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.240404 16327 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:41.240698 11408 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:41.234964 11514 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:41.242645 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.245394 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.231575 16344 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.250349 16082 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.251593 16193 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:41.252512 15885 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.534s	user 0.007s	sys 0.011s
I20260709 13:16:41.254192 15885 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.256654 15885 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.257942 15953 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.263202 16327 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:41.263718 16327 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.264374 16327 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.265261 15940 raft_consensus.cc:3055] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:41.265739 15940 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:41.267921 16077 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:41.268419 16077 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:41.269459 16077 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.270567 16077 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.274278 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.274996 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.275290 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.276459 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:41.266737 15940 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.228484 16336 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.288836 15852 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.248322 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.263701 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:41.276415 16192 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.005s
W20260709 13:16:41.281409 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.267884 16204 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.281854 11367 raft_consensus.cc:606] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Received request to transfer leadership
I20260709 13:16:41.282204 16293 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.282503 15953 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:41.283392 11364 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:16:41.263902 15970 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.291620 11367 raft_consensus.cc:614] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:41.296958 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:41.294745 15852 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.298705 15852 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:41.299263 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 55545.4us
I20260709 13:16:41.300246 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:41.301502 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 75303.3us
I20260709 13:16:41.302194 11334 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.302505 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.303153 16293 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.294625 16192 consensus_queue.cc:1243] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:41.304068 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:41.302608 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1155) took 3073 ms. Trace:
I20260709 13:16:41.307634 12069 rpcz_store.cc:276] 0709 13:16:38.229046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.234593 (+  5547us) service_pool.cc:224] Handling call
0709 13:16:39.281379 (+1046786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:39.709055 (+427676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.301389 (+1592334us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b5400741dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:41.302225 (+   836us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400741dd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:41.302418 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3368700544}
I20260709 13:16:41.308943 16075 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:41.309523 16075 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:41.295658 15970 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.311359 15852 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.313364 16207 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.305697 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.310606 16075 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.305065 15430 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.317157 15940 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.303200 16068 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.306568 16204 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:41.323200 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 1981) took 1433 ms. Trace:
I20260709 13:16:41.323678 11369 rpcz_store.cc:276] 0709 13:16:39.889429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.909775 (+ 20346us) service_pool.cc:224] Handling call
0709 13:16:39.910054 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.593192 (+683138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.322779 (+729587us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400423454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:41.323046 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469410560}
I20260709 13:16:41.325071 16326 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.325635 16326 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.327541 15810 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:41.328871 16213 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.325204 16193 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:41.330561 16193 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.330930 16193 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:41.331604 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2549) took 1387 ms. Trace:
I20260709 13:16:41.332146 16335 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.332751 16335 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.334919 16335 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.335870 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.336477 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:41.336925 11441 raft_consensus.cc:2410] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:41.337522 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:41.338235 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.339711 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.342254 15698 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.342494 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.343274 16092 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.343938 16092 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.344364 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.344724 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.305454 16293 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.346210 16092 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.346580 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:41.346721 16092 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.342847 16162 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.347668 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.348328 12066 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:41.347996 16162 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.349260 16134 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.349545 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.351644 16129 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:41.352195 16129 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.354396 11443 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:41.302870 15936 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:41.354952 16129 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.355908 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.356357 12034 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.356510 12066 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:41.348124 16221 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.357512 16221 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.358923 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.358009 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.359179 16245 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.361943 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.360863 11510 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 13:16:41.364077 12066 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:41.363687 16245 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.362365 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.365177 16245 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.366470 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:41.366928 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.357297 16219 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.354413 16205 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.368108 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:41.368474 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.368336 16205 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.361292 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.369513 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.369726 11361 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:41.347136 11444 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.370393 12061 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:41.371336 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.372164 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.350956 16162 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.372787 12061 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:41.347318 16092 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.374388 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.375010 16092 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.363631 11365 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:41.331990 11439 rpcz_store.cc:276] 0709 13:16:39.943787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.946235 (+  2448us) service_pool.cc:224] Handling call
0709 13:16:39.946563 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:41.045174 (+1098611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.331312 (+286138us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54003138d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:41.331548 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145401728}
I20260709 13:16:41.375761 16245 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.372494 15698 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.376636 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.377553 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.376901 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.378731 15698 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.365445 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.379527 12066 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:41.354568 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.378494 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.375782 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.329751 11512 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
W20260709 13:16:41.382112 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 44670.4us
I20260709 13:16:41.386507 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.370304 16134 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.388010 16302 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:16:41.388454 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 93411.8us
I20260709 13:16:41.388681 16337 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:41.381091 11370 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:41.398229 11444 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:41.388938 16134 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.438742 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.442883 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:41.429121 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 72828.7us
W20260709 13:16:41.447105 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 57489.1us
I20260709 13:16:41.359696 16221 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.448874 16213 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.450796 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.473393 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:41.391067 16207 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.391709 16337 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.389102 16303 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.388515 16302 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.451361 11362 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:41.480150 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:41.454722 16129 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:41.456988 16174 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.458951 16335 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.459342 16205 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.461454 16306 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.467880 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:41.473979 12063 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:16:41.496290 15810 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:41.496697 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.496887 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.497135 16250 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.497545 16250 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.497862 12063 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:41.498216 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.498659 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.498796 12064 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:41.496918 16192 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.499531 15936 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:41.497875 16250 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.500535 15936 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.502702 15936 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.473409 12067 raft_consensus.cc:1217] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[8.8-12.10]   Dedup: 8.8->[10.9-12.10]
I20260709 13:16:41.503739 12067 pending_rounds.cc:85] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 8
I20260709 13:16:41.503978 12067 pending_rounds.cc:107] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:16:41.504247 12067 raft_consensus.cc:2889] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:41.451871 16185 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.506534 16207 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.499223 12063 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:41.507210 16207 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.479112 15953 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.391482 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:41.506342 16185 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.516547 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:41.508703 11439 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:16:41.510309 11517 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:41.510214 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.509359 16207 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.467226 11513 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:41.511597 16186 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:41.485885 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.512161 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.512921 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.507547 12068 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:41.511297 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 43613us
I20260709 13:16:41.500156 16192 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.522763 12062 raft_consensus.cc:2368] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14.
I20260709 13:16:41.522480 11519 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:41.488821 16337 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:41.485137 16302 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:41.522460 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.526510 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.530025 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.530826 11436 raft_consensus.cc:2410] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:16:41.546476 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.528962 11441 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:41.547297 11512 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:41.529606 12060 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:41.472978 16193 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.546554 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.529469 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.554596 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.562206 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.564944 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:41.567471 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 265421us
I20260709 13:16:41.564258 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:41.568317 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.491652 16213 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.557973 11440 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:41.572214 11441 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:41.572207 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:41.577507 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.579545 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.583596 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.574039 11513 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:41.554778 16207 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.520679 15953 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.579975 11439 raft_consensus.cc:2376] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8. Re-sending same reply.
I20260709 13:16:41.587376 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:41.589859 11515 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:41.591455 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.581857 11445 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:41.590137 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.572028 16221 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.526938 16327 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.599813 11436 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:41.579762 16162 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:41.604928 11519 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:41.605832 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.598464 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:41.606737 11443 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:41.600826 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.609601 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.610257 11439 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 8.
I20260709 13:16:41.611070 11442 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:41.620636 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.563728 16335 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.590521 16075 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.617435 11438 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:41.599009 16205 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.630781 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.635761 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:41.636839 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 70919.4us
I20260709 13:16:41.638271 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:41.641480 11436 raft_consensus.cc:2410] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:16:41.616604 16155 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:41.652412 11510 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:41.623370 16162 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:41.609381 15970 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.593825 16207 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.657262 11513 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:41.626097 16186 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:41.678565 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:41.618791 16221 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.652647 15850 raft_consensus.cc:2764] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:16:41.682873 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.687690 11441 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:41.683008 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.660249 16344 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:41.683188 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:41.682749 11333 leader_election.cc:400] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:16:41.686685 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:41.688699 11362 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:41.694674 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.740048 15901 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:41.740756 15901 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:41.741741 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 101848us
I20260709 13:16:41.743279 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:41.743811 16162 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:41.744570 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 143800us
I20260709 13:16:41.746857 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.690104 16134 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:41.747843 15970 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.748896 16106 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:41.749468 16106 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.749933 11368 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:41.740885 15760 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:41.751957 15760 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.752357 16335 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.686861 16326 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.680184 16328 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.754765 15760 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.754663 16328 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:41.755734 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 105878us
I20260709 13:16:41.756784 16328 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.757838 16250 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.759627 16186 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.719326 16344 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.761459 16344 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:41.763381 16347 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:16:41.764922 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 109204us
I20260709 13:16:41.765635 11365 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:41.766520 16185 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.672158 16075 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:41.748068 16134 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:41.741382 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.768292 11364 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:41.768409 16134 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.770454 16207 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:41.770923 11262 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56588 (request call id 238) took 13245 ms (client timeout 15000 ms). Trace:
I20260709 13:16:41.677662 16205 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:41.772085 11266 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 8 to 13, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:41.773921 16174 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.774928 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:41.775604 11369 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:41.779314 12061 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:41.781450 16192 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.782796 15953 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.636930 15936 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:16:41.768882 16075 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:41.784144 16327 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:41.818557 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 75331.2us
I20260709 13:16:41.818753 15970 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:41.796510 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 49695.3us
I20260709 13:16:41.803825 12068 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:41.802172 11362 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:41.802693 11519 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:41.786432 11445 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:41.805624 11368 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:41.825676 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:41.824999 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 186780us
I20260709 13:16:41.830631 12032 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:41.772840 16205 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:41.828792 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 72294.8us
I20260709 13:16:41.826368 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:41.835619 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:41.838246 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:41.843560 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 254604us
I20260709 13:16:41.838867 11442 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:41.847434 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:41.849635 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 83639.1us
I20260709 13:16:41.851060 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:41.817174 16229 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:41.851938 12033 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:41.799898 16306 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.857501 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:41.859515 15901 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:41.861606 12069 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:41.832197 15970 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.774153 11266 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 9 to 10, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 10 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:41.872108 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:41.859905 16193 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:41.872720 11442 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:41.856496 15928 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:41.855667 16192 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:41.866588 16178 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:41.849840 15885 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:41.885114 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.863741 16174 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:41.883584 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:41.868582 15970 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.864110 16207 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:41.899677 11266 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 10, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 10 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:41.875576 15940 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:41.869501 15852 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.896440 12033 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:41.974201 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 90674.2us
I20260709 13:16:41.883175 16214 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:41.992465 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.890548 16068 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.993494 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.993748 11519 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:41.891191 16213 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.001929 11513 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:41.925833 16174 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:42.005223 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:41.929416 16178 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:42.014915 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.014791 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:42.016386 11515 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:42.016564 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.014762 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:41.885975 16293 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.135s
I20260709 13:16:41.934669 15885 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.021799 16293 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:42.022241 16293 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:42.023249 16293 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.023814 15885 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.024541 16293 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:42.025199 16328 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:42.025799 16328 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.026415 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.027086 11517 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:42.027866 11367 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:42.015290 16174 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:42.030668 11368 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:42.031674 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.032253 11362 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:41.995497 16214 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:42.032752 16214 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:41.771341 11262 rpcz_store.cc:269] 0709 13:16:28.525586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.526350 (+   764us) service_pool.cc:224] Handling call
0709 13:16:41.770689 (+13244339us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:37.166946 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:37.587779 (+420833us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:37.596538 (+  8759us) write_op.cc:435] Acquired schema lock
0709 13:16:37.601217 (+  4679us) tablet.cc:662] Decoding operations
0709 13:16:37.606921 (+  5704us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:37.607016 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:37.607042 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:16:37.608317 (+  1275us) tablet.cc:701] Row locks acquired
0709 13:16:37.608360 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:16:37.608521 (+   161us) write_op.cc:270] Start()
0709 13:16:37.608698 (+   177us) write_op.cc:276] Timestamp: P: 1783602997608461 usec, L: 0
0709 13:16:37.608716 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:16:37.782349 (+173633us) log.cc:844] Serialized 5116 byte log entry
0709 13:16:38.493735 (+711386us) op_driver.cc:464] REPLICATION: finished
0709 13:16:38.962805 (+469070us) write_op.cc:301] APPLY: starting
0709 13:16:38.962909 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:39.411880 (+448971us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:39.411988 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:39.500521 (+ 88533us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:39.500814 (+   293us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:16:39.500862 (+    48us) write_op.cc:312] APPLY: finished
0709 13:16:39.502092 (+  1230us) log.cc:844] Serialized 121 byte log entry
0709 13:16:40.623213 (+1121121us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:40.637219 (+ 14006us) write_op.cc:454] Released schema lock
0709 13:16:40.654294 (+ 17075us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:40.676984 (+ 22690us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:40.677012 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5394566,"reported_tablets":43,"spinlock_wait_cycles":727296,"child_traces":[["op",{"apply.queue_time_us":468821,"apply.run_cpu_time_us":80946,"apply.run_wall_time_us":1709448,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58415,"num_ops":13,"prepare.queue_time_us":129973,"prepare.run_cpu_time_us":24006,"prepare.run_wall_time_us":1119181,"raft.queue_time_us":106746,"raft.run_cpu_time_us":42665,"raft.run_wall_time_us":324792,"replication_time_us":884860,"thread_start_us":1898229,"threads_started":5,"wal-append.queue_time_us":1761346,"wal-append.run_cpu_time_us":7739,"wal-append.run_wall_time_us":2119623}]]}
I20260709 13:16:42.039331 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.040179 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.042215 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.043358 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.043613 16335 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:42.044013 11366 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:42.045097 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:42.044798 16178 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.045710 11363 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:41.874986 16330 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.146s
I20260709 13:16:42.039620 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.052075 11444 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:16:42.054136 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 61610us
I20260709 13:16:42.044180 11518 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:42.064088 11515 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:42.039685 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.060906 11443 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:42.070029 11438 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:42.073130 11363 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
W20260709 13:16:42.058327 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 191584us
I20260709 13:16:42.063228 11266 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.025312 16363 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:42.098791 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:42.099758 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.103497 11364 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:42.044153 16335 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:42.109133 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 212761us
I20260709 13:16:42.110069 12033 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:42.114104 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:42.094151 11266 catalog_manager.cc:5697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 10, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 10 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:41.993466 15852 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:41.992420 16192 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:42.120548 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 65862.1us
I20260709 13:16:42.115084 11333 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:42.127410 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:42.124481 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 503904us
I20260709 13:16:42.128173 11443 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:42.104197 15953 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:16:42.116441 16337 consensus_meta.cc:287] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.003s
I20260709 13:16:42.088009 16207 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:16:42.100272 16068 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:42.129966 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 243787us
W20260709 13:16:42.192355 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 77361.2us
I20260709 13:16:42.136348 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:42.252565 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 66732us
I20260709 13:16:42.351593 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:42.358055 11483 leader_election.cc:400] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14.
I20260709 13:16:42.361840 11483 leader_election.cc:403] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:16:42.194741 11266 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 7, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:42.123239 16328 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:42.359889 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:42.135061 16302 consensus_meta.cc:287] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.000s
I20260709 13:16:42.128397 16293 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:42.358511 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:42.127344 15852 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:42.358333 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:42.128428 16306 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:42.464633 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:42.466521 16330 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.466761 16372 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:16:42.464130 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 112748us
W20260709 13:16:42.451740 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 104924us
I20260709 13:16:42.358654 16337 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:42.452401 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 93567.6us
I20260709 13:16:42.475732 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.476022 11437 raft_consensus.cc:2393] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:42.477281 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:42.477398 16204 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:42.478495 16197 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:16:42.478883 16378 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.205s	user 0.003s	sys 0.000s
I20260709 13:16:42.452193 11266 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.453372 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:42.453651 16068 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:42.131930 16155 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:42.451155 15698 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:42.454346 16192 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.121661 16363 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.128582 16331 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.327s
I20260709 13:16:42.467124 16372 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:42.463788 11441 raft_consensus.cc:2368] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12.
I20260709 13:16:42.460805 16302 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.464186 16213 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:42.467693 11367 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:42.480110 16335 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.466400 16229 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Leader election lost for term 13. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14.
I20260709 13:16:42.530081 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:42.136353 16344 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.487582 16328 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:42.488317 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.489487 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:42.483487 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:42.534034 11439 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:42.492038 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:42.495950 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:42.496977 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:42.452024 16368 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:16:42.500298 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.500639 11266 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 9 to 10, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 10 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.479907 16197 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:16:42.475734 16337 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:42.513762 12061 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:16:42.467095 16330 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.477836 16174 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:42.526224 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:42.495791 15698 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.491310 15885 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:16:42.532709 12063 raft_consensus.cc:2368] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:16:42.532800 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:42.534740 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51363.2us
I20260709 13:16:42.550132 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.541205 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:42.192432 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 56956.3us
I20260709 13:16:42.501637 16250 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:42.513258 16213 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:42.543882 12068 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:42.546180 16174 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:42.551424 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:42.556728 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:42.552078 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:42.534772 16068 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:42.552606 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:42.553632 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:42.533625 16328 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:42.536716 16335 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:42.559062 11366 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:42.575227 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2711) took 1014 ms. Trace:
I20260709 13:16:42.527812 16214 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.584807 11439 raft_consensus.cc:2410] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:16:42.588769 11445 rpcz_store.cc:276] 0709 13:16:41.561250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.609860 (+ 48610us) service_pool.cc:224] Handling call
0709 13:16:42.559262 (+949402us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b5400278dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:42.575127 (+ 15865us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129188224}
I20260709 13:16:42.607342 15698 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:42.608353 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51715us
I20260709 13:16:42.609269 12033 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:42.609633 11441 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:42.609978 16250 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:42.610088 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:42.610463 16250 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.610682 16373 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:42.610895 16250 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:42.611155 16373 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.611562 16373 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:42.614713 16214 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.619808 11441 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:42.620887 11266 catalog_manager.cc:5697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 5 to 8, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:42.556859 16213 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:42.685083 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 206916us
W20260709 13:16:42.635809 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 85975.6us
I20260709 13:16:42.687268 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:42.559783 16068 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:42.584383 16328 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:42.688853 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:42.695642 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:42.690469 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 80454.4us
I20260709 13:16:42.689208 11266 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 6 to 10, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 10 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:42.694152 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:42.588088 16335 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.694350 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:42.604249 15885 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:42.705974 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:42.706635 12069 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:42.707430 11439 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:16:42.709398 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.712005 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.712391 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.712630 11364 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:42.713205 11514 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:42.715461 11370 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:42.715698 12061 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:16:42.715860 16037 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.716418 16037 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.717353 16331 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:42.718510 16037 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:42.708935 15885 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:42.710108 16092 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:16:42.721086 15970 consensus_meta.cc:287] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.747s	user 0.003s	sys 0.005s
I20260709 13:16:42.722263 16221 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.704212 16382 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.728299 11369 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:42.726673 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.683455 15810 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:42.759852 11364 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 14: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:42.723385 16092 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:42.725528 15970 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.800450 11513 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:42.800860 11443 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:42.764503 16376 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:16:42.764722 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.734301 16155 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:42.735268 16382 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.769016 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.806097 11518 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:42.723591 16229 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:16:42.785543 11266 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 11 to 14, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 14 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:42.707252 16335 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:42.636466 16372 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.783216 16363 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.725773 16302 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.724334 15885 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:42.821017 16155 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:42.789736 15810 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:42.854838 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.849411 11266 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:42.860025 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 150664us
I20260709 13:16:42.868841 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.869597 11515 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:42.870581 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.871169 11515 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:42.873380 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:42.850117 16344 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:42.879052 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:42.880008 12061 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:16:42.881202 11266 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 12 to 13, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 13 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:42.881165 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:42.886786 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:42.887321 12067 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:42.888002 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:42.889147 11266 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 8 to 12, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 12 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:42.875983 16330 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.894057 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1064) took 1368 ms. Trace:
I20260709 13:16:42.890911 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:42.894413 11510 rpcz_store.cc:276] 0709 13:16:41.525591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.578470 (+ 52879us) service_pool.cc:224] Handling call
0709 13:16:41.578769 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:41.655847 (+ 77078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.893816 (+1237969us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b540014d7d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:42.893989 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168964352}
W20260709 13:16:42.890650 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 186180us
I20260709 13:16:42.890667 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:42.896116 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 180005us
I20260709 13:16:42.895219 12060 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:16:42.896752 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:42.898437 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:42.899421 16178 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:42.900794 11266 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 9, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:42.900646 12034 leader_election.cc:400] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12.
I20260709 13:16:42.904933 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:42.905311 12034 leader_election.cc:403] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:16:42.905259 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:42.913136 11439 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:42.891669 15970 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:42.916442 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:42.921058 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:42.902558 16330 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:42.923964 16330 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:42.924393 16330 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:42.883002 16092 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.928022 11266 catalog_manager.cc:5697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.930105 11266 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 12, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 12 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:42.917732 16178 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:42.931952 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:42.932073 16399 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:42.933488 15430 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:42.934139 15885 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:42.936287 16328 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:42.943349 11316 auto_rebalancer.cc:893] Removing replace marker failed: Replica 01ae50c877d04ba687e4e06259ba5581 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:42.947759 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 51891.6us
W20260709 13:16:42.949355 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:42.966161 11316 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:16:42.906484 16389 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:42.913942 15810 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:42.953377 16398 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:42.953429 16396 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:42.955008 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:42.922403 16385 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:16:42.955224 16335 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:42.964619 11266 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.966739 12067 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:42.968544 12034 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:42.966569 11316 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
W20260709 13:16:42.969360 16392 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:16:42.979985 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:42.985720 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:42.987298 11362 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:42.987751 11361 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:42.987130 11445 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:42.966133 15430 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.989107 11266 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:42.916440 16221 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:16:42.965970 16399 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:42.968232 16389 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:42.995803 11519 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:42.968729 16376 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:16:43.006608 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.007339 11445 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:42.975596 16396 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:42.978853 16371 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12.
I20260709 13:16:42.972270 15885 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:42.992271 15430 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:42.995935 15852 consensus_meta.cc:287] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
I20260709 13:16:42.975982 16197 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:16:42.995299 16037 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:16:43.005330 16382 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:42.994074 16399 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:42.985414 16398 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.006951 16344 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:42.997179 16389 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.050176 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.062749 11445 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:43.011938 15940 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:42.993111 16250 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.044322 15810 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.042361 16396 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:43.069294 15940 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.047900 15970 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:43.080860 15940 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.081444 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.082651 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.084952 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.087710 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.088418 11444 raft_consensus.cc:2393] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:43.090516 16398 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.090656 16335 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:43.091463 15970 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.092677 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.091871 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2553) took 1506 ms. Trace:
I20260709 13:16:43.093473 11438 rpcz_store.cc:276] 0709 13:16:41.585626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.660623 (+ 74997us) service_pool.cc:224] Handling call
0709 13:16:42.069855 (+409232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.086674 (+ 16819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.091690 (+1005016us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54003151d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:43.091817 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904269696}
I20260709 13:16:43.047803 15852 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.062956 16382 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:43.098227 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2535) took 1069 ms. Trace:
I20260709 13:16:43.089200 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.099937 11517 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:43.098654 11367 rpcz_store.cc:276] 0709 13:16:42.029271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.029369 (+    98us) service_pool.cc:224] Handling call
0709 13:16:42.029617 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.513492 (+483875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.097985 (+584493us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400165b54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:43.098160 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980317568}
I20260709 13:16:43.053517 16397 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:16:43.105329 16248 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:43.105544 16397 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:43.105778 16248 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:43.105945 16397 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.106077 15885 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.106635 16248 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.111475 16382 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.102624 16330 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.113775 16382 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.086949 16155 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.114007 15885 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:43.153733 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 185278us
I20260709 13:16:43.114787 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:43.135221 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 149042us
I20260709 13:16:43.131930 15970 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:43.126067 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 139413us
I20260709 13:16:43.153595 15852 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.114287 16330 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.073005 16178 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.162621 16328 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.163580 15953 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.163728 16406 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:43.114845 15885 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.164975 16336 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:43.165339 16336 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:43.166366 15885 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.166132 16336 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.169881 16336 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:43.157063 16373 consensus_meta.cc:287] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.003s
I20260709 13:16:43.176946 11438 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:43.164096 15953 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:43.181880 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 226958us
I20260709 13:16:43.179878 11334 leader_election.cc:400] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:16:43.182926 11334 leader_election.cc:403] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:16:43.182778 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:43.128906 16408 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:43.182178 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:43.157455 16248 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.185339 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.189155 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:43.163625 16366 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.889s	user 0.009s	sys 0.000s
I20260709 13:16:43.191354 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:43.194789 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.195710 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.196578 11444 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:43.198172 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.190285 16330 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:16:43.199169 12066 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:43.199550 11367 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:43.199186 11362 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:16:43.199697 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 232658us
I20260709 13:16:43.202873 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.201996 11334 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.153532 16213 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.203802 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:43.205348 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.204042 11445 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:43.206679 12065 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:43.206779 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.207350 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.209473 12063 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:43.209501 12064 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:43.186772 16106 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.212212 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.212107 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.204358 16330 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:43.200270 16328 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:43.214200 15940 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.217568 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.224877 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:43.215816 12065 raft_consensus.cc:2376] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15. Re-sending same reply.
I20260709 13:16:43.197530 16221 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:16:43.200184 16371 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:43.199882 16092 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:43.229854 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.184614 16373 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.228049 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.231956 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:43.214751 16386 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.461s	user 0.001s	sys 0.001s
I20260709 13:16:43.214612 16036 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:16:43.232125 11484 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.233700 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:43.234375 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.233008 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.236577 12063 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:43.233256 11441 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:43.230664 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:43.217813 16331 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:43.241349 16331 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:43.241801 16331 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.242484 16213 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.235541 15885 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:16:43.243427 16377 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:43.243752 15885 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:43.244426 15885 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:43.244743 15885 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:43.245450 15885 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.246584 15885 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:43.233009 16415 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:43.248473 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.203222 15430 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.231465 16406 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:43.253366 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:43.254104 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.252281 16335 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:16:43.259217 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:43.250424 16376 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:16:43.252413 16204 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:43.261153 16204 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.261494 16204 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.260802 11370 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:43.247884 16398 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:43.248186 16371 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.267514 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:43.267915 12067 raft_consensus.cc:2393] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:43.252554 16250 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.275702 16412 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:43.276038 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1011) took 1714 ms. Trace:
I20260709 13:16:43.263471 16415 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:43.255214 16406 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.287552 16365 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:43.292775 11484 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:43.287889 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:43.266615 16398 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:43.295455 16398 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.297089 16373 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:43.298283 15430 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.301234 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.302778 16335 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:43.303184 16335 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.303570 16335 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:43.303789 16404 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:43.307108 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:43.308403 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.281296 11436 rpcz_store.cc:276] 0709 13:16:41.561531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.650814 (+ 89283us) service_pool.cc:224] Handling call
0709 13:16:41.670570 (+ 19756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:41.921908 (+251338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:41.921940 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:42.014389 (+ 92449us) log.cc:844] Serialized 26 byte log entry
0709 13:16:42.445635 (+431246us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:42.445679 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.446017 (+   338us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.214115 (+768098us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:43.275760 (+ 61645us) raft_consensus.cc:1685] finished
0709 13:16:43.275781 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.275980 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36207,"raft.queue_time_us":140092,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"spinlock_wait_cycles":9278720,"thread_start_us":420560,"threads_started":2,"wal-append.queue_time_us":495232}
I20260709 13:16:43.250329 16396 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.315539 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.312510 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:43.316165 11444 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.291831 16406 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.321904 16389 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.234925 16397 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:43.323719 16106 consensus_queue.cc:1175] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6e87b213a4554d94825aceec352fc094" responder_term: 12 status { last_received { term: 11 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:43.325410 16389 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.306285 16412 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:43.331635 11367 raft_consensus.cc:1240] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:43.310654 16221 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.314327 16336 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:43.323815 16397 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.318625 16393 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:43.287766 16371 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.346053 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.347723 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.352022 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.353051 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.354511 11444 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:43.352567 12060 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:43.305822 16365 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.356926 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.334064 16412 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.358873 12064 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:43.337514 16229 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:43.355841 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.337410 16336 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.336249 16398 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.359187 16412 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:43.363600 16229 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:43.361685 11334 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:43.364997 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 52589us
I20260709 13:16:43.365168 11444 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.365829 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:43.368907 16082 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:43.369287 16082 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.369611 16082 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:43.356804 16221 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.372751 16221 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.373312 11440 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
W20260709 13:16:43.372973 11437 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2710) took 1865 ms (client timeout 1745 ms). Trace:
W20260709 13:16:43.374228 11437 rpcz_store.cc:269] 0709 13:16:41.515987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.553694 (+ 37707us) service_pool.cc:224] Handling call
0709 13:16:43.372741 (+1819047us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54000843d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:43.372878 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093583104}
I20260709 13:16:43.374955 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.375514 11439 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.346582 16393 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.376839 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.363273 16336 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:43.378489 11445 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
W20260709 13:16:43.380755 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.382733 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:43.383621 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.385505 16404 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:43.386097 16365 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.385459 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.387432 12063 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:43.375329 16036 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:43.390113 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.390020 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:43.390777 11436 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:43.391623 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.393062 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.394134 11444 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:43.392580 15698 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:43.395766 15885 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:43.388505 16229 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.396474 15885 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.396760 15698 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.397617 15698 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.399466 16174 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.396585 15817 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:16:43.403604 11408 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.553s (SENT) [suppressed 6 similar messages]
I20260709 13:16:43.389737 15940 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.403759 15817 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.388568 16036 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.404413 15817 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:43.405076 11518 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:16:43.380434 16393 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:43.412184 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2468) took 2159 ms. Trace:
I20260709 13:16:43.425530 16365 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:43.426102 16365 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.428582 16365 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.392709 16404 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:43.415413 15940 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:16:43.433544 16331 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.412442 16036 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.438477 16204 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.438927 11333 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.440773 16248 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.387660 16278 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:16:43.444577 11510 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:43.437685 11518 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:43.416131 11511 rpcz_store.cc:276] 0709 13:16:41.234085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.239569 (+  5484us) service_pool.cc:224] Handling call
0709 13:16:41.351596 (+112027us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b5800010b98. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19d7fc0 00007f35c19c2929 00007f35c19c23d2 00007f35baf39bd4
0709 13:16:41.351770 (+   174us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:41.352217 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:41.520532 (+168315us) log.cc:844] Serialized 14 byte log entry
0709 13:16:41.665534 (+145002us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:41.684920 (+ 19386us) log.cc:844] Serialized 52 byte log entry
0709 13:16:41.685347 (+   427us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:41.685716 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 13:16:41.686163 (+   447us) log.cc:844] Serialized 14 byte log entry
0709 13:16:41.686382 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.718602 (+ 32220us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:41.778259 (+ 59657us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b18002ab5c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:43.087438 (+1309179us) spinlock_profiling.cc:243] Waited 8.26 ms on lock 0x7b48001b0cd8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c1927aa1 00007f35c191eb37 00007f35c191e9ca 00007f35c1a1626f 00007f35c1a15f37 00007f35c1a2a55c 00007f35c1a2a508 00007f35c1a2a488 00007f35c1a2a443 00007f35c1a296ec 00007f35c0953a96
0709 13:16:43.163028 (+ 75590us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:43.327573 (+164545us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b18002ab5c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:43.392843 (+ 65270us) raft_consensus.cc:1685] finished
0709 13:16:43.392864 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.392984 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49297,"prepare.queue_time_us":355363,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":109,"raft.queue_time_us":3713,"raft.run_cpu_time_us":25980,"raft.run_wall_time_us":1504703,"spinlock_wait_cycles":368608640,"thread_start_us":250533,"threads_started":2,"wal-append.queue_time_us":711200}
I20260709 13:16:43.444008 16396 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.449787 11439 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:43.468948 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:43.469969 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 66396.9us
I20260709 13:16:43.466034 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:43.385443 16068 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.475466 16068 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.475762 16204 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.473939 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:43.480399 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.483945 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:43.482093 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.486177 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:43.487617 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2561) took 1557 ms. Trace:
I20260709 13:16:43.487934 11442 rpcz_store.cc:276] 0709 13:16:41.930409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.930575 (+   166us) service_pool.cc:224] Handling call
0709 13:16:42.559478 (+628903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.585803 (+ 26325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.585834 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:42.585883 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:42.585920 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.585982 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.487460 (+901478us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b5400278dd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:43.487564 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1316681088}
I20260709 13:16:43.472214 16423 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.491812 11365 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 14: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:43.464346 16335 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.492759 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.493254 11514 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.494144 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:43.494342 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:43.488039 15698 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:43.492267 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:43.496336 11364 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.470539 16397 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 13:16:43.499536 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.484715 16162 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:43.484794 15940 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:43.501817 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.472388 16092 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:16:43.508862 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.488124 16370 consensus_queue.cc:1059] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 13:16:43.520613 16422 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:16:43.524250 16155 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.525897 16398 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.499416 16423 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.538983 16399 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.495434 16036 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.548329 11514 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:43.527343 16406 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.550307 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.550909 16248 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:43.551391 16406 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:43.528828 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.531598 11364 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:43.515599 15817 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.550848 11514 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.504240 15940 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:43.573302 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.553102 11362 raft_consensus.cc:2368] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:16:43.573963 11442 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.575467 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.576083 11439 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.515038 15885 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:43.582028 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.510775 16162 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.589715 11439 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
W20260709 13:16:43.591068 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 1012) took 1452 ms (client timeout 1644 ms). Trace:
I20260709 13:16:43.539013 16423 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.592796 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.593211 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.594038 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.604607 11445 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.604816 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:43.594461 16174 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:43.595333 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1366) took 1762 ms. Trace:
W20260709 13:16:43.600728 11443 rpcz_store.cc:269] 0709 13:16:42.138861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.185906 (+ 47045us) service_pool.cc:224] Handling call
0709 13:16:43.590878 (+1404972us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54002ea754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:43.591000 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792981632}
I20260709 13:16:43.592576 15970 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:43.606515 11514 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:43.579393 15940 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.610433 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.612241 11514 raft_consensus.cc:2393] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:43.605562 12068 rpcz_store.cc:276] 0709 13:16:41.832920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.833088 (+   168us) service_pool.cc:224] Handling call
0709 13:16:41.833370 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:42.555200 (+721830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.595087 (+1039887us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54001ff9d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:43.595275 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633633664}
I20260709 13:16:43.613744 11443 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:16:43.591964 15852 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.610803 15885 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:43.576210 16365 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:16:43.616384 16213 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:43.615485 16155 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:43.640103 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.647977 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.603758 15953 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:43.658807 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.658656 11445 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:43.634455 15936 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:43.616942 16336 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.657858 16155 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.669593 16412 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.719022 16365 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.693457 15430 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:43.695796 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2555) took 2054 ms. Trace:
I20260709 13:16:43.689970 11367 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 6. Current term is 7. Ops: []
I20260709 13:16:43.703697 15936 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.732949 15430 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.702073 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:43.670864 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.674846 16178 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:43.678128 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.669787 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.672196 16306 consensus_queue.cc:1059] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:43.740826 15953 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.741042 16306 raft_consensus.cc:3055] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:43.741400 16306 raft_consensus.cc:740] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.742501 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.676537 11443 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:43.698947 16036 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.678366 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:43.679890 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.735529 11440 rpcz_store.cc:276] 0709 13:16:41.641610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:41.719052 (+ 77442us) service_pool.cc:224] Handling call
0709 13:16:43.373137 (+1654085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:43.377780 (+  4643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.695597 (+317817us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54000843d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:43.695739 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3803339008}
I20260709 13:16:43.702723 16372 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.703557 16399 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.711723 16106 raft_consensus.cc:3055] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:16:43.687654 16429 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:16:43.699828 15817 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.717306 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.695540 16162 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.720679 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.686280 16415 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:43.730118 11442 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:43.762157 11517 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:43.762573 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.762923 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.636662 15885 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:43.639238 16213 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:43.730880 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.734977 11441 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:43.736737 12064 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:43.736425 11514 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:43.732690 16214 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:16:43.738541 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.739413 11436 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:43.742201 16306 consensus_queue.cc:260] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.700150 11368 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:43.746424 11519 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:43.746697 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:43.747556 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 45573.7us
I20260709 13:16:43.748582 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.749024 11512 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:43.742435 15936 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.749543 16398 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:43.756714 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.758716 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.763211 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.841957 11517 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:43.842967 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.844580 11517 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:43.845342 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.843436 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.775763 15953 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:16:43.860390 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:43.864604 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 284371us
I20260709 13:16:43.763386 11513 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:16:43.711786 16370 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:16:43.734967 16219 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.765002 11444 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:43.766112 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:43.881453 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 115443us
I20260709 13:16:43.882817 16092 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:43.882592 15885 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.885313 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.886379 12033 leader_election.cc:400] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:16:43.886700 12033 leader_election.cc:403] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:16:43.888824 16155 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:43.883373 16092 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.889966 16092 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.891817 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.892486 11513 raft_consensus.cc:2368] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:43.893198 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.894375 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:43.857062 16373 consensus_queue.cc:1059] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:16:43.804329 11442 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:43.827294 16082 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:43.922976 16082 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:43.881682 11408 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:43.897624 16213 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.924058 11518 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:43.896104 16306 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:43.827888 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:43.775377 16404 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:43.928066 16404 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:43.928972 16404 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.930516 16335 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:43.827257 16278 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:43.931825 16278 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:43.826136 12065 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.829293 16371 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:43.932080 16331 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.830088 16336 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:43.757771 16106 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.932154 16278 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:43.837563 11440 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:43.737002 16178 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:43.793285 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.745106 16365 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.896723 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:43.939188 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 111376us
I20260709 13:16:43.826393 11443 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.897028 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.946733 11515 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:43.899907 11364 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:43.863423 16248 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Committing config change with OpId 8.8: config changed from index 4 to 8, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:43.877249 15940 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.863087 15698 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:16:43.906381 16393 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.907995 16412 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:43.887789 16219 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.912516 15852 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:43.912957 16174 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.901460 12064 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:43.918764 11516 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:16:43.897516 16068 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:43.932533 16371 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.991883 12068 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 4 to 8, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:43.943017 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:43.944708 11445 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:16:43.945466 11442 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:43.998529 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.998873 11443 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:44.000474 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:43.999500 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:44.001310 11442 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:43.997071 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:44.003835 11445 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:43.964100 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:43.968128 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.970427 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:43.981529 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.008260 11443 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:44.009416 12069 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:44.004594 15430 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:43.981499 16213 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:43.964275 16082 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:43.980666 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 221998us
I20260709 13:16:44.012583 11517 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:44.020004 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.014575 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:43.983754 16446 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.023727 11436 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:44.024709 11440 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:44.027129 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:44.029858 11442 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:43.980103 16335 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:16:44.044435 11515 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:44.047600 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.045847 11445 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:44.026721 15817 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:44.051144 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:44.051357 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.056370 12061 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:44.051594 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.052865 12032 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.015143 16036 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:44.057952 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 89765.2us
I20260709 13:16:44.043746 16214 consensus_queue.cc:1243] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:44.060144 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.060952 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.061810 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.077718 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:44.078790 11510 raft_consensus.cc:2368] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14.
I20260709 13:16:44.080165 12068 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:44.007894 16174 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.083616 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.047886 16412 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:44.015313 16155 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:16:43.990749 16106 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:44.116365 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 50897.2us
I20260709 13:16:44.114619 16365 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:44.114815 11544 heartbeater.cc:646] Failed to heartbeat to 127.10.4.126:42409 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.10.4.126:42409 timed out after 15.000s (SENT)
I20260709 13:16:44.107650 15810 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:44.022872 16373 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 13:16:44.112191 16425 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:44.119903 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.054493 16293 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:44.035207 16446 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.046700 16411 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:16:44.078496 16092 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:44.132548 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.127002 11511 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:44.108974 16036 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.128393 16219 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.010381 16278 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:44.128712 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.141590 11510 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 17: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:44.129199 16241 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.130139 16174 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.095563 16214 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:44.123210 15810 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.132902 16365 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.132164 16456 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:44.159044 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 107521us
I20260709 13:16:44.159555 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.161247 11362 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:44.161568 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.148409 15810 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.146669 16214 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:44.163228 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 143286us
I20260709 13:16:44.156739 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.170480 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.173538 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.176230 11510 raft_consensus.cc:2393] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:44.148474 16293 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.183676 11335 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.169050 16219 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.163483 15953 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:44.175941 16214 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.189777 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.188346 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.186501 16459 consensus_queue.cc:1059] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:16:44.195266 16459 raft_consensus.cc:3055] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:44.195684 16459 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:44.195904 16442 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:44.196516 16459 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.197680 16459 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:44.199370 16293 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.200374 16440 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:44.201164 16450 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:44.201565 16450 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:44.202356 16450 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.150838 16241 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.160625 16365 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.159652 16451 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:16:44.186301 16444 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:16:44.214926 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 156267us
I20260709 13:16:44.231827 11362 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:44.185073 16452 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:44.201138 16440 raft_consensus.cc:2764] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:16:44.292241 16444 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:16:44.278201 11510 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:44.278524 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45535.9us
I20260709 13:16:44.277015 11408 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:44.214118 16219 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.193063 16393 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.321482 16293 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:44.338635 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.337239 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1021) took 1160 ms. Trace:
I20260709 13:16:44.274927 16447 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:16:44.386721 11518 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:44.324699 16452 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:44.293097 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 240295us
I20260709 13:16:44.278800 16412 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:44.343636 16446 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.203851 15953 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.302455 16464 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:44.278566 16450 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.364427 11519 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
W20260709 13:16:44.364070 16437 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.505s	user 0.004s	sys 0.001s
W20260709 13:16:44.386758 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 109820us
I20260709 13:16:44.388465 16468 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:44.393052 16468 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:44.393254 11408 leader_election.cc:400] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:44.393754 11408 leader_election.cc:403] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:16:44.310348 12065 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
W20260709 13:16:44.323283 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 149909us
I20260709 13:16:44.388701 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.290033 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:44.395738 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 105788us
I20260709 13:16:44.386729 11438 rpcz_store.cc:276] 0709 13:16:43.176645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.193290 (+ 16645us) service_pool.cc:224] Handling call
0709 13:16:43.193497 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:44.162443 (+968946us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:44.162471 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:44.162511 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:44.162560 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.336976 (+174416us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b54003133d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:44.337025 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:44.337170 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37728,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":2213820672}
I20260709 13:16:44.390272 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:44.391323 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.398413 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.399165 16440 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:44.401495 16468 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.400792 16155 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:44.405225 16465 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:44.405795 16465 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:44.406616 16465 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.408290 11335 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.408634 16440 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:44.409531 11335 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:44.372444 15810 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:44.412843 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.388818 16082 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:44.417248 16219 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:44.417841 16449 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:44.395352 16397 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:16:44.423976 11438 pending_rounds.cc:85] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 10
I20260709 13:16:44.454152 11438 pending_rounds.cc:107] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:16:44.454428 11438 raft_consensus.cc:2889] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:44.424142 16365 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.456260 11444 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:44.456591 11362 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:16:44.458413 11370 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:44.392055 16293 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.408703 16241 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.427275 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:44.427610 11510 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:44.427541 12065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:44.461045 12065 raft_consensus.cc:606] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.461391 12065 raft_consensus.cc:614] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:44.382469 16214 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.427816 16398 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:44.436177 16371 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:44.438491 16454 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.285s	user 0.001s	sys 0.001s
I20260709 13:16:44.439006 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.391077 16464 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:44.439850 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.441008 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.441506 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.416601 15817 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:44.444227 11511 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
W20260709 13:16:44.449208 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 53491.4us
I20260709 13:16:44.452823 11437 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:44.479773 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.456185 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.480681 11368 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:44.424825 16036 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.465418 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:44.469934 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.472344 11513 raft_consensus.cc:2368] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:44.486204 16336 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:44.486757 16336 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.487736 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.488807 16336 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.453747 16444 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:44.474793 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.475733 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:44.505643 12067 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:44.476732 12060 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:44.477231 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:44.479818 11518 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:44.481191 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.455937 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.484383 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.484717 15817 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:44.494055 11366 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:44.486132 11362 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:44.518802 11511 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:44.517356 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:44.519393 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:44.483582 16241 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:16:44.467088 16092 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:44.504765 16468 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:44.469338 16382 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:16:44.526440 16444 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:44.545463 16440 raft_consensus.cc:2764] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:16:44.483795 16082 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:16:44.536314 11514 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 13:16:44.548306 11368 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:16:44.548727 11368 pending_rounds.cc:85] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 7
I20260709 13:16:44.548947 11368 pending_rounds.cc:107] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:16:44.549160 11368 raft_consensus.cc:2889] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:44.551573 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.538635 11511 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:44.556304 16214 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.557114 11514 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:44.559641 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.489837 16412 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.559710 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.561079 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.550269 16241 consensus_queue.cc:1243] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:44.516388 16452 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.564484 11362 raft_consensus.cc:1240] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:44.522454 16365 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.579456 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.565380 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.566176 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.548348 16464 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.556353 16293 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.583081 15810 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.545673 16444 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.584360 16464 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:44.586863 16248 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:34564 (request call id 16) took 4297 ms. Trace:
W20260709 13:16:44.566637 16473 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:16:44.579574 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.559495 11364 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:44.592870 11370 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:16:44.593724 11364 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:44.594331 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.596915 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.597612 11442 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:44.598688 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.598873 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:44.600096 11437 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:44.597620 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.600808 11444 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:44.603055 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.603663 11362 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:44.604163 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.603765 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.523375 15817 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.607728 11436 raft_consensus.cc:2955] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 4 to 8, VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:44.603693 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.611004 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1907) took 1165 ms. Trace:
I20260709 13:16:44.611375 11516 rpcz_store.cc:276] 0709 13:16:43.445209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.445339 (+   130us) service_pool.cc:224] Handling call
0709 13:16:44.610859 (+1165520us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400064ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:44.610953 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888826624}
I20260709 13:16:44.616906 11516 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:44.627061 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.624639 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.630142 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:44.632063 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.581105 15953 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:44.632367 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:44.635655 11369 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:16:44.637133 11409 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.636049 11516 raft_consensus.cc:1240] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:44.639323 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.642051 11368 raft_consensus.cc:2393] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:44.649178 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.653018 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.653723 11335 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:44.654697 11516 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:44.628597 16459 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.635999 16214 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.632717 16450 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:44.636708 15810 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.648082 16394 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:44.657737 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:44.640344 16082 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:44.640615 16468 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:44.645802 16393 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:44.661139 11516 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:44.663229 15940 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.661468 11517 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:44.644807 16365 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:16:44.661804 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1925) took 1115 ms. Trace:
I20260709 13:16:44.671772 11513 rpcz_store.cc:276] 0709 13:16:43.546018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.963858 (+417840us) service_pool.cc:224] Handling call
0709 13:16:44.661612 (+697754us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54003d9354. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:44.661749 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072886656}
I20260709 13:16:44.654098 15953 consensus_queue.cc:1059] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:16:44.636705 16241 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:16:44.627938 16467 consensus_queue.cc:1059] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:16:44.657182 16440 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:44.659694 15936 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:44.664772 16452 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.664563 16450 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:44.681762 15953 raft_consensus.cc:3055] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:44.682538 16082 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:44.687461 11517 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:44.690635 11517 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:44.675727 15885 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:44.587261 16248 rpcz_store.cc:276] 0709 13:16:40.288980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:40.289156 (+   176us) service_pool.cc:224] Handling call
0709 13:16:41.174805 (+885649us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400232420. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc97b520 00007f35bca04b42 00007f35bca03800 00007f35bc9fc9fa 00007f35c19c44c9 00007f35baf40c54 00007f35baf40be2 00007f35baf40b11 00007f35baf40a8c 00007f35baf3fd42
0709 13:16:41.213289 (+ 38484us) log.cc:844] Serialized 487 byte log entry
0709 13:16:44.586741 (+3373452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2926,"raft.queue_time_us":23154,"raft.run_cpu_time_us":1307,"raft.run_wall_time_us":5361,"spinlock_wait_cycles":478882304,"thread_start_us":10027,"threads_started":1,"wal-append.queue_time_us":60496,"wal-append.run_cpu_time_us":1949,"wal-append.run_wall_time_us":1001997}
I20260709 13:16:44.660308 16459 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.677094 16468 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:44.651319 16178 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.680156 15936 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:44.666370 15810 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.688802 16450 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:44.706254 16399 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.707562 16457 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:44.707909 16399 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.708074 16483 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:16:44.709127 16483 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:44.709440 16483 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:44.709941 16399 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:44.711874 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 268492us
W20260709 13:16:44.713152 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.714607 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1086) took 1006 ms. Trace:
I20260709 13:16:44.715143 11510 rpcz_store.cc:276] 0709 13:16:43.707762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.518351 (+810589us) service_pool.cc:224] Handling call
0709 13:16:44.527084 (+  8733us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.527296 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.527318 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.534007 (+  6689us) log.cc:844] Serialized 26 byte log entry
0709 13:16:44.664171 (+130164us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:44.664206 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.664267 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:44.714410 (+ 50143us) raft_consensus.cc:1685] finished
0709 13:16:44.714430 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:44.714552 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":101041,"threads_started":1,"wal-append.queue_time_us":151664}
I20260709 13:16:44.716789 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 857937fa61dc400a81345944f0a14511 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:16:44.717231 16376 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:44.717631 16376 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:44.700688 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:44.694195 11511 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:44.688613 15953 raft_consensus.cc:740] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:44.678222 16423 consensus_queue.cc:1059] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:16:44.694343 11517 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.714787 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.702644 15936 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.724567 11441 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:44.726493 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:44.727326 11517 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:44.729166 15817 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:44.729588 15817 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.729923 15817 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:44.733986 11517 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:44.737001 15817 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.718456 16376 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.741192 16376 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:44.719096 16178 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:44.740156 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:44.746227 11441 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:44.747889 16490 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:44.748307 16490 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:44.749131 16490 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.750304 16490 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.747913 15953 consensus_queue.cc:260] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.757179 15953 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:44.760035 16486 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:44.761492 16459 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.768425 15810 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.770200 16468 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.747270 16241 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:44.771466 16241 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.771802 16241 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:44.779125 11443 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:16:44.773751 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.780406 11436 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:44.783114 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.783286 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:44.799273 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.799891 11517 raft_consensus.cc:2376] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12. Re-sending same reply.
W20260709 13:16:44.800987 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 54379us
I20260709 13:16:44.802364 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.802934 11517 raft_consensus.cc:2376] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13. Re-sending same reply.
I20260709 13:16:44.802959 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.805146 11510 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:16:44.710131 16483 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.796272 15810 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:44.805951 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.811834 11517 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:16:44.807307 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.810317 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:44.836133 11511 raft_consensus.cc:2376] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14. Re-sending same reply.
I20260709 13:16:44.830811 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.830927 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.804955 16423 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:16:44.838227 11513 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:44.832510 15936 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.841991 15936 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.834214 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:44.843645 11368 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:44.843961 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 1090) took 1135 ms. Trace:
I20260709 13:16:44.844352 11519 rpcz_store.cc:276] 0709 13:16:43.708735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.054539 (+345804us) service_pool.cc:224] Handling call
0709 13:16:44.843816 (+789277us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54003d8954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:44.843904 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415502720}
I20260709 13:16:44.800578 15817 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.835323 11484 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:44.838896 11510 raft_consensus.cc:2393] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:44.810672 15810 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:44.837002 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.852416 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:44.852990 11436 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.832502 16278 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.855566 15810 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:44.858584 16371 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:44.859616 16398 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.860095 16398 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.862007 16398 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:44.863648 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.865624 11369 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:44.866660 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:44.866915 16373 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:44.867276 11362 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:44.867484 16373 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.863533 16483 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:44.868924 11512 raft_consensus.cc:2410] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:44.869566 16373 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:44.870466 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 1082) took 1163 ms. Trace:
I20260709 13:16:44.870819 11512 rpcz_store.cc:276] 0709 13:16:43.706794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.049032 (+342238us) service_pool.cc:224] Handling call
0709 13:16:44.870273 (+821241us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54001513d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:44.870413 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858147072}
I20260709 13:16:44.874864 11516 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:16:44.875216 11516 pending_rounds.cc:85] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 8
I20260709 13:16:44.875451 11516 pending_rounds.cc:107] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:16:44.875677 11516 raft_consensus.cc:2889] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:44.876313 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:44.879346 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 350349us
I20260709 13:16:44.880841 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:44.893348 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.908422 11436 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:44.872921 16467 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:44.911283 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:44.913484 11335 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:44.917918 11445 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:16:44.919095 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:44.919760 11333 leader_election.cc:400] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14.
I20260709 13:16:44.921126 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:44.917721 16373 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:44.922914 16373 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.925305 16373 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:44.930143 16450 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.931433 16495 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:44.885515 16397 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:44.941963 16397 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.885456 16278 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:16:44.955060 16491 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:44.955674 16491 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:44.956518 16491 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.879138 16155 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.957655 16155 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:44.959685 16068 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:44.960299 16467 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:44.955801 16404 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:44.961220 16467 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.961863 16404 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:44.933786 16250 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:44.938078 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:44.966814 11440 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
W20260709 13:16:45.067759 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.040424 11445 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:45.226171 11366 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:16:44.987089 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 66020us
I20260709 13:16:45.222273 11368 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:16:45.184016 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:45.089057 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.211010 15430 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:44.992760 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.885682 15885 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:45.234551 11333 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:45.231251 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.100049 12063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:45.074339 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:45.236073 12063 raft_consensus.cc:606] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:44.985569 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:44.984495 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:45.213212 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 294187us
I20260709 13:16:44.983706 11441 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:45.214844 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:45.233757 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.238315 12060 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:45.238696 11444 raft_consensus.cc:2393] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 13: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:45.186017 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1918) took 1641 ms. Trace:
I20260709 13:16:45.228827 12067 raft_consensus.cc:2410] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:16:45.240983 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:45.241837 12067 raft_consensus.cc:2368] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:45.239487 11518 rpcz_store.cc:276] 0709 13:16:43.544363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.390731 (+846368us) service_pool.cc:224] Handling call
0709 13:16:44.479644 (+ 88913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:44.509500 (+ 29856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.185817 (+676317us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b5400065ed0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:45.185957 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203831040}
W20260709 13:16:45.236493 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 162250us
I20260709 13:16:45.246287 12069 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:45.243130 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:45.184072 16436 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:16:45.248003 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:44.918524 16219 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.079963 16495 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.234086 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.193269 16106 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:44.959697 16155 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.095664 15953 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:44.960269 16068 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:45.291520 16389 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:45.292073 16389 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:45.293004 16389 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.293671 16436 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:45.294478 16241 consensus_meta.cc:287] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.001s
I20260709 13:16:45.183662 16393 raft_consensus.cc:3055] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:44.985365 16394 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:45.295071 16394 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:45.296082 16394 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.297242 16394 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:45.297837 16219 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.299719 16393 raft_consensus.cc:740] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:45.254235 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:45.300458 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:45.300858 12060 raft_consensus.cc:2368] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:45.300631 16393 consensus_queue.cc:260] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.252228 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:45.292938 16068 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:45.304747 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 71186us
I20260709 13:16:45.304949 12033 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:45.305295 16483 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:45.307708 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:45.308282 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:45.309926 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 78686.9us
I20260709 13:16:45.310539 12064 raft_consensus.cc:2368] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:45.301616 16393 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:45.309250 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:45.249660 16411 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:44.944439 16397 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.250326 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:45.145186 16450 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.066926 15852 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:45.016438 16250 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.066776 15810 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:44.947885 16423 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.313s
I20260709 13:16:45.285102 16492 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.289656 15953 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.294188 16389 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:45.294227 16436 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.311430 11366 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:16:45.291133 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:45.274209 11484 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:45.018672 15817 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:45.316226 11370 raft_consensus.cc:1240] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:16:45.182068 16036 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:45.251070 16491 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.272055 16412 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.323678 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:45.341694 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:45.341979 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 67924.9us
I20260709 13:16:45.243856 16371 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.307785 15940 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:45.340157 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:45.353770 12068 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:16:45.331447 16469 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:16:45.317025 16396 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.355832 11369 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:16:45.351526 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:45.341547 16036 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:45.282791 16162 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:16:45.329644 16411 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:45.311975 16370 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:16:45.309664 16495 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:45.377337 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.368067 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:45.294957 16241 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.376048 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:45.379086 11512 raft_consensus.cc:2368] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:45.378336 11366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:45.380167 11366 raft_consensus.cc:606] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:45.381309 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:45.380118 12033 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:45.396121 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:45.328819 16464 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:16:45.341003 15817 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:45.019089 16404 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.381223 16411 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:45.319024 16082 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:16:45.355003 15885 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:45.353116 15940 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.405061 16306 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:45.373656 16396 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.337074 16492 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.416177 16499 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:45.487562 12067 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:45.425545 15810 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.363008 16245 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.427433 16436 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.428541 16502 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:45.431951 16250 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.433754 16241 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.437702 15817 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:45.373888 16423 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.441042 16036 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.352790 16491 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.331776 15852 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.482299 16396 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:45.364552 16371 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.443856 16404 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:45.353067 16412 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.474840 15940 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:45.504663 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 263380us
I20260709 13:16:45.520569 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:45.572929 11515 raft_consensus.cc:2410] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:45.522676 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:45.485107 15885 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:45.567936 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:45.570770 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:45.570961 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:45.571861 11517 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:45.503618 15953 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.506336 16436 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:45.573654 11513 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:45.578600 11510 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:45.494338 16479 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
W20260709 13:16:45.586202 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.586833 16336 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:45.587157 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:45.587654 15885 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:45.588653 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.590549 11437 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:16:45.590667 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:45.591322 11438 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
W20260709 13:16:45.586244 16498 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:16:45.591881 11444 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:45.592983 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:45.577114 16450 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:45.514014 16423 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.599529 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:45.613560 11513 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:45.577914 16036 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:45.606995 11517 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:45.597712 11519 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:45.606449 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:16:45.607558 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 295393us
I20260709 13:16:45.606281 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.617151 11438 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:45.503831 16245 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.601639 16336 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:45.610713 11437 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:16:45.610958 16479 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:45.596203 16436 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:45.618875 11515 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:16:45.621380 11408 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:45.585424 16468 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:16:45.629539 11408 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:45.635131 11510 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:45.622995 16508 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:16:45.637034 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 283502us
I20260709 13:16:45.643994 16492 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:45.648404 16423 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.649053 11513 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:45.638218 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:45.634271 16495 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.611929 16457 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:16:45.661010 16457 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:45.661474 16457 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:45.625154 16479 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:45.632745 15953 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:45.615126 16450 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.627661 16241 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 13:16:45.678654 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 306249us
I20260709 13:16:45.679574 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:45.632972 16082 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:16:45.636554 16467 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:16:45.662328 16457 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.704600 16479 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.705782 16479 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:45.690073 16250 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:45.692750 16510 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:16:45.719928 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 323588us
W20260709 13:16:45.720166 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.724506 16245 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.697302 16336 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.726027 16412 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:45.715854 16523 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.728003 16450 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.728564 16371 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.752411 16523 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.754750 16523 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.703272 16512 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:45.764479 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2590) took 1168 ms. Trace:
I20260709 13:16:45.764883 11443 rpcz_store.cc:276] 0709 13:16:44.596098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.603552 (+  7454us) service_pool.cc:224] Handling call
0709 13:16:45.764325 (+1160773us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54003156d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:45.764424 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399426304}
W20260709 13:16:45.767256 16497 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 0.934s	user 0.003s	sys 0.004s
I20260709 13:16:45.770020 15953 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:45.750756 16483 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:45.777565 16483 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.778061 16483 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.779088 16241 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:45.779489 16241 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:45.780105 16483 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.780308 16241 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.707623 16513 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.781936 16513 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.783929 16513 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.788754 16510 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:45.789702 16510 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.798450 16512 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:45.742081 16219 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:45.726401 15852 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:45.726020 16306 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:16:45.868090 16250 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:45.867107 15940 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.748605 15817 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.693593 16373 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:16:45.817518 11366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:45.872049 11366 raft_consensus.cc:606] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:45.872393 11366 raft_consensus.cc:614] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:16:45.874812 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 261932us
W20260709 13:16:45.882287 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 162491us
I20260709 13:16:45.897046 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1097) took 1411 ms. Trace:
I20260709 13:16:45.897651 11516 rpcz_store.cc:276] 0709 13:16:44.485592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.669598 (+184006us) service_pool.cc:224] Handling call
0709 13:16:44.871654 (+202056us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:44.884378 (+ 12724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.896770 (+1012392us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001513d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:45.896962 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438460672}
W20260709 13:16:45.899334 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 271893us
I20260709 13:16:45.870666 15940 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:45.911199 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:45.917713 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:45.922231 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 331621us
W20260709 13:16:45.919109 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:45.925585 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.926420 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:45.931155 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.932060 11443 raft_consensus.cc:2410] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:16:45.934281 11441 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:45.937755 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:45.939854 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 205501us
I20260709 13:16:45.941824 11441 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: []
W20260709 13:16:45.944505 16505 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.359s	user 0.003s	sys 0.003s
I20260709 13:16:45.944888 11441 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:45.947856 16515 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:45.948365 16515 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.948694 16515 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:45.949437 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:45.952008 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.956457 16515 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:45.958647 16393 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:45.959147 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2578) took 2231 ms. Trace:
I20260709 13:16:45.959642 16394 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:45.960072 16394 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:45.960892 16394 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:45.961748 16241 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:45.962494 16394 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:45.964985 16515 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:45.969606 16504 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.400s	user 0.004s	sys 0.003s
I20260709 13:16:45.970682 16412 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:45.971197 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1054) took 1090 ms. Trace:
I20260709 13:16:45.971581 11442 rpcz_store.cc:276] 0709 13:16:44.880749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.880887 (+   138us) service_pool.cc:224] Handling call
0709 13:16:44.881205 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:44.881592 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.881618 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:44.922432 (+ 40814us) log.cc:844] Serialized 52 byte log entry
0709 13:16:45.225988 (+303556us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:45.226032 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.226104 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.968914 (+742810us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:45.970874 (+  1960us) raft_consensus.cc:1685] finished
0709 13:16:45.970896 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.971137 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":23524,"spinlock_wait_cycles":215655296,"thread_start_us":302961,"threads_started":1,"wal-append.queue_time_us":552881}
I20260709 13:16:45.939302 16336 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:45.976579 11442 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:45.978904 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.979631 11442 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:45.980389 11441 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:16:45.980676 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.984112 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.984784 11441 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:16:45.959591 11439 rpcz_store.cc:276] 0709 13:16:43.727495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.727648 (+   153us) service_pool.cc:224] Handling call
0709 13:16:43.728005 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:44.134433 (+406428us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:44.388346 (+253913us) log.cc:844] Serialized 14 byte log entry
0709 13:16:44.564719 (+176373us) log.cc:844] Serialized 14 byte log entry
0709 13:16:44.564973 (+   254us) log.cc:844] Serialized 14 byte log entry
0709 13:16:44.565045 (+    72us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:44.625192 (+ 60147us) log.cc:844] Serialized 52 byte log entry
0709 13:16:44.625351 (+   159us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:44.625559 (+   208us) log.cc:844] Serialized 14 byte log entry
0709 13:16:44.625797 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.186874 (+561077us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b1800180220. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:45.352929 (+166055us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b1800180220. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419
0709 13:16:45.371311 (+ 18382us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800180220. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:45.958789 (+587478us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b540027abd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:45.958847 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.958895 (+    48us) raft_consensus.cc:1685] finished
0709 13:16:45.958909 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.959064 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4359,"prepare.queue_time_us":1449290,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":8211,"raft.queue_time_us":172735,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":26608,"spinlock_wait_cycles":648580992,"thread_start_us":489281,"threads_started":3,"wal-append.queue_time_us":170673}
I20260709 13:16:45.989215 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.989568 16491 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:45.989794 11439 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:45.992321 12066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1225) took 1057 ms. Trace:
I20260709 13:16:45.992820 12066 rpcz_store.cc:276] 0709 13:16:44.934508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.951919 (+ 17411us) service_pool.cc:224] Handling call
0709 13:16:44.952291 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.350240 (+397949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.350271 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.350309 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:45.350344 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.992085 (+641741us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54005212d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:45.992135 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.992261 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914775424}
I20260709 13:16:45.992965 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:45.997587 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.000536 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:46.007341 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.009155 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:45.923673 16377 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:16:46.014189 15760 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.025018 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.023699 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.026011 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.015939 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:46.026705 11440 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:46.026386 12065 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:46.009490 11369 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:46.017261 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.017602 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.017261 11364 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:45.919068 16459 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:16:46.031282 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.033177 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:45.902576 16411 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.034499 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:46.035809 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 152738us
I20260709 13:16:46.036556 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:46.032667 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.043659 11366 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:46.022631 16512 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.009828 11370 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:45.923453 16483 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:46.029904 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:46.050563 11364 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.050935 11370 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:46.018842 16328 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.028584 11361 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:46.019351 16376 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.033277 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.026779 11365 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:46.051261 11370 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:46.054246 12069 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 7. Current term is 8. Ops: []
I20260709 13:16:46.056911 16328 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.057982 16511 raft_consensus.cc:993] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:46.062729 11364 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:46.067683 11370 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.069244 11370 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:46.044948 16502 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.044677 16499 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.046358 16092 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.075922 15430 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:46.077399 15430 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:46.035223 15760 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.053409 12033 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:46.070461 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 47261.3us
I20260709 13:16:46.053068 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:46.051224 16483 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:46.019654 16221 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.101095 16248 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:46.103571 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.070114 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.071897 16479 raft_consensus.cc:1081] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:46.122334 15430 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:46.125212 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 487046us
I20260709 13:16:46.098308 16306 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:16:46.101999 16483 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:46.132248 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.054976 16512 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.144510 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:46.117920 11366 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:16:46.072697 16376 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.153599 11444 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:46.118724 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.118389 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:46.119371 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.101590 16221 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.155539 12067 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:46.117545 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.119673 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.156912 12061 raft_consensus.cc:2410] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:16:46.117949 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.157668 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:46.147202 15430 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.148612 12064 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:46.147933 12068 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:46.110088 16508 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:46.114646 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44593.7us
I20260709 13:16:46.152735 11442 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:46.021911 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:46.094903 16502 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.168051 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.168715 11409 leader_election.cc:400] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:46.168795 16412 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:46.169605 16502 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:16:46.095104 16499 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.169739 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.169905 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.170570 16412 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:46.171538 16499 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.114391 16520 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.172412 16520 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.174389 16520 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:46.114065 16373 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:46.161016 16529 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.175534 16336 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:46.175678 16529 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.176082 16336 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.177809 16529 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.160918 16519 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:46.178413 16519 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.180474 16519 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.165587 16467 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:46.165835 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:46.181588 16467 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.166412 16404 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:46.175040 16373 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.183136 16404 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:46.183529 16404 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.183867 16404 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:46.184274 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:46.184806 16373 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.158118 16397 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:46.185786 16397 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.186546 15698 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:46.186908 15698 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:46.167011 16532 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:16:46.187778 16397 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.167073 16492 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:46.182098 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:46.185788 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 133536us
I20260709 13:16:46.178447 16278 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:46.189013 16278 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.191074 16278 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.187693 15698 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.186106 16306 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:46.198465 16306 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.198873 16306 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.199460 11365 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:46.199951 16389 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:46.199733 16404 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.182005 16467 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:46.126669 16530 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:46.202631 16404 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:46.203969 15698 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:46.205636 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.190801 12067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:46.206164 12067 raft_consensus.cc:606] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:46.190856 16412 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:46.207151 16412 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.207545 16412 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:46.208078 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.210175 16376 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.191243 16519 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:46.156436 15817 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.212687 12068 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:16:46.193715 16328 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.194418 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.177946 16336 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.213423 16436 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.194676 15885 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:46.195413 16512 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.196622 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.220676 11361 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 7.
I20260709 13:16:46.196768 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.196960 16221 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.192696 15760 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.197808 16411 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:46.097695 16092 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:46.214986 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 193169us
I20260709 13:16:46.218753 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.224664 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.225374 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.197314 12033 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:46.222144 11408 leader_election.cc:400] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:46.227247 11408 leader_election.cc:403] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:16:46.229130 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.230257 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.230361 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.230994 12064 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:16:46.232103 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.233495 12061 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:46.234764 12061 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:46.236366 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.237811 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.238458 12060 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 7.
I20260709 13:16:46.268771 11370 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:16:46.269253 11370 pending_rounds.cc:85] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 8
I20260709 13:16:46.269502 11370 pending_rounds.cc:107] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:16:46.272631 11370 raft_consensus.cc:2889] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:46.221680 11366 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:46.225299 12063 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:46.107245 16248 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.284773 11436 raft_consensus.cc:2393] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 17: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:16:46.285578 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 88342.9us
I20260709 13:16:46.290549 12033 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:46.214741 16467 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.292568 16092 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.293618 16467 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.284403 16532 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:46.294442 16532 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:46.294788 16532 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:46.214370 16394 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:16:46.297081 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.297704 11445 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 6.
I20260709 13:16:46.298708 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.301266 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.301920 11436 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:46.304677 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:46.286819 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:46.287859 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 60847.3us
I20260709 13:16:46.305871 12063 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
W20260709 13:16:46.288597 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 49597.5us
I20260709 13:16:46.307621 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:46.312795 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:46.162801 16459 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:46.315894 12068 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:46.318706 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.318118 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:46.320704 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.322052 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:46.322181 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 152516us
I20260709 13:16:46.322870 11409 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.323804 11365 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:46.325619 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:46.326077 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 105852us
I20260709 13:16:46.322876 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:46.326920 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:46.326874 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:46.328150 11436 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:46.328826 16404 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:46.325387 16519 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:46.330299 16535 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:46.330771 16389 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:46.328837 16534 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.331207 16389 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.331655 16389 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:46.329226 16404 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.333118 16404 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:46.333511 16376 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.333969 16376 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.335927 16508 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.337594 16529 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:46.338598 16036 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:16:46.338937 16036 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:46.341625 16512 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:46.325558 16398 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.342672 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:46.345742 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:46.342355 16398 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.342257 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:46.310101 16533 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.351846 11436 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 18.
I20260709 13:16:46.352521 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:46.349426 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.299814 16396 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:46.346416 11510 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:46.352875 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:46.240206 16483 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.316933 16467 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:46.361109 12062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.007s	sys 0.023s
W20260709 13:16:46.361464 12062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.007s	sys 0.023s
I20260709 13:16:46.362248 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:46.363771 11409 leader_election.cc:400] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:46.364547 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1214) took 1047 ms. Trace:
I20260709 13:16:46.364663 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.365935 11409 leader_election.cc:400] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:46.366701 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.326745 16336 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:46.343677 16411 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:46.375269 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.368635 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.380517 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.377938 11334 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:46.385736 11436 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:46.386194 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:46.381485 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.343570 16465 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:16:46.351900 16068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.011s	sys 0.020s
I20260709 13:16:46.366843 11368 rpcz_store.cc:276] 0709 13:16:45.316797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.317747 (+   950us) service_pool.cc:224] Handling call
0709 13:16:45.318082 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.318284 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:45.574460 (+256176us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.362580 (+788120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.362631 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:46.362674 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.362739 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.364501 (+  1762us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87791,"prepare.queue_time_us":408933,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":30,"raft.queue_time_us":94015,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"thread_start_us":940411,"threads_started":3,"wal-append.queue_time_us":570183}
W20260709 13:16:46.390257 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.390893 11369 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:46.386766 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.344324 16512 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:46.392506 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.327541 16459 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.392692 11436 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:46.368381 16278 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:46.406746 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.397789 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:46.399619 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:46.378477 15953 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:46.385222 16411 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.383780 16219 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:46.401417 11335 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:46.390996 16465 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.389724 16479 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:46.392359 15885 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.405400 16377 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:46.408179 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.363624 16535 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:16:46.389688 16068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.011s	sys 0.022s
I20260709 13:16:46.350013 16457 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:16:46.358129 16513 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:16:46.409555 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:46.398717 16036 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.408704 11484 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:46.413252 11485 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.003s	sys 0.046s
I20260709 13:16:46.411693 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 931) took 1075 ms. Trace:
I20260709 13:16:46.400199 16512 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.510486 12062 rpcz_store.cc:276] 0709 13:16:45.336500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.336699 (+   199us) service_pool.cc:224] Handling call
0709 13:16:45.338260 (+  1561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.338489 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:45.708640 (+370151us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.394762 (+686122us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.409154 (+ 14392us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.411280 (+  2126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.411331 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:46.411382 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.411520 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.411635 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127928,"prepare.queue_time_us":148586,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":96,"raft.queue_time_us":56142,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":5080,"spinlock_wait_cycles":8122624,"thread_start_us":992041,"threads_started":2}
I20260709 13:16:46.382591 16467 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.398554 16396 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:46.410204 16411 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:46.363361 16533 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.418701 11514 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:46.506192 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.413426 16155 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:46.508435 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 98939.8us
I20260709 13:16:46.415730 15817 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:46.417380 16465 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:46.416181 16479 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.418727 16106 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:46.514199 11366 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 8. Current term is 9. Ops: []
I20260709 13:16:46.417959 16219 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.505553 16241 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.515762 16519 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:46.543038 16245 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:46.516719 16036 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.562005 16219 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:46.519018 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 110398us
I20260709 13:16:46.563864 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.519991 16492 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:46.520401 16278 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.567575 16278 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.568238 16306 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.543622 15817 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:46.520015 16539 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:16:46.419690 16377 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:16:46.512521 11485 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.003s	sys 0.046s
I20260709 13:16:46.449707 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.575706 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:46.489202 15953 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:46.497145 16515 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:46.532130 16248 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.536051 16396 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.536813 16398 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.498910 16376 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.539719 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.541332 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.542253 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.501319 15885 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.592995 15885 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.552489 11445 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
W20260709 13:16:46.516156 16540 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:16:46.576851 11436 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 6.
I20260709 13:16:46.594162 16377 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.550052 16479 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:46.580976 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:46.595301 11361 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.595546 16377 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:46.595942 16467 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.587127 11368 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:46.582273 15430 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:46.599223 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.558473 16241 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.573926 15817 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.550384 16535 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:46.600454 16535 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.570219 16278 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.586748 12062 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:46.602468 16535 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.570263 16306 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:46.584070 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.594497 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:46.604956 11436 raft_consensus.cc:3060] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.607210 11361 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:46.607286 16241 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.610289 11368 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:46.612182 12033 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:46.612843 16479 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.613274 15430 raft_consensus.cc:2804] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:46.613758 16306 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:46.599828 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
W20260709 13:16:46.626554 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 85281.3us
I20260709 13:16:46.627686 11440 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:46.620927 15430 raft_consensus.cc:697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:46.628876 15430 consensus_queue.cc:237] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.631901 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.621737 11436 raft_consensus.cc:2468] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:46.633047 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:46.636194 11439 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.033s	sys 0.041s
I20260709 13:16:46.637605 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:46.583043 16245 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:46.632938 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:46.586807 16460 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:46.610406 16436 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:46.643512 12060 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:46.653167 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:46.643230 11439 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.033s	sys 0.041s
I20260709 13:16:46.586721 16457 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:46.618111 16515 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:46.626897 15885 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.594946 16412 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:46.700335 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 325110us
I20260709 13:16:46.708015 16513 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.707419 11443 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:46.709375 16532 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.726308 16532 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:46.727320 16559 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:16:46.727861 16515 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.626197 16535 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:16:46.655055 15817 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.741882 15817 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.743603 16241 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:46.745095 16155 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:46.745486 16155 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:46.746443 16155 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.695755 16457 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.747779 16457 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.752223 11370 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:46.754998 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:46.713702 16411 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.714022 16534 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:46.646823 16467 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:46.718284 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.707302 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:46.765770 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 58546.8us
I20260709 13:16:46.706984 16519 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.724066 16513 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.654282 16460 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:46.760262 12034 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:46.761507 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.761615 11370 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 18.
I20260709 13:16:46.698902 15852 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.706768 16536 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:46.757939 16306 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:46.769990 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 51791us
I20260709 13:16:46.762244 16499 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:46.757664 16370 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:46.792608 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:46.768615 16241 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:46.715159 15885 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.774102 16155 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:46.852458 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.770840 16511 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.076s
W20260709 13:16:46.843264 16404 consensus_meta.cc:287] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.000s
I20260709 13:16:46.886457 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:46.774377 16459 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.878753 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:46.800046 16522 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:46.807107 16499 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.799505 16467 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.896332 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:46.798892 15852 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.773911 15817 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:16:46.806841 16306 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.880213 16533 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:46.877137 16241 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.883059 16396 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:46.842823 16370 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.775563 16460 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:46.897081 16522 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.901396 16561 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:46.908901 16535 raft_consensus.cc:2764] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:16:46.902118 16404 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:46.905663 16565 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:46.911307 16036 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.971629 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:46.972409 11438 raft_consensus.cc:2368] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 17. Current term is 18.
I20260709 13:16:46.905596 16499 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:46.913954 16566 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.977174 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:46.775220 16519 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:46.967684 16389 consensus_meta.cc:287] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.631s	user 0.007s	sys 0.002s
I20260709 13:16:46.959429 16522 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:47.002719 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.003906 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:47.005112 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.006126 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:47.013612 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 105540us
I20260709 13:16:47.014386 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:46.952263 16492 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:46.969008 16479 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:46.913482 16467 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.007099 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:46.958675 16241 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:46.960361 16533 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:16:47.055275 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.055856 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.060607 11512 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:47.057478 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:46.962018 16562 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:16:47.062340 11511 raft_consensus.cc:2368] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14.
I20260709 13:16:47.060370 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:47.062352 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 285387us
W20260709 13:16:47.042658 16556 log.cc:927] Time spent T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:16:46.959669 16396 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.066246 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.068478 11515 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:46.994633 16036 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.052073 16561 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:47.076630 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.052460 16336 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:47.078796 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.079965 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.080574 11511 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:47.013485 16389 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.059764 16412 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.083144 11335 leader_election.cc:400] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 17. Current term is 18.
I20260709 13:16:47.086509 11335 leader_election.cc:403] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:16:47.008908 16571 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:16:47.065894 16411 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:47.083953 11484 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:47.084205 11438 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:47.067315 16532 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.008805 16566 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.067499 16479 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:47.077095 16512 consensus_meta.cc:287] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
I20260709 13:16:47.108062 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:47.076776 16561 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:47.121212 16561 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:47.130167 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:47.130220 11335 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:47.131151 11438 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:47.131716 11366 raft_consensus.cc:2393] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 8: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:16:47.133291 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49425.5us
I20260709 13:16:47.110018 16219 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 17. Current term is 18.
I20260709 13:16:47.142925 16590 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.146766 16590 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.117377 16412 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:47.134378 16555 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:16:47.128230 16565 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.212102 16565 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.212641 16565 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.213985 16561 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.214768 16565 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.216284 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2749) took 1213 ms. Trace:
I20260709 13:16:47.216642 11439 rpcz_store.cc:276] 0709 13:16:46.003310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.003466 (+   156us) service_pool.cc:224] Handling call
0709 13:16:46.003770 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.004316 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.004343 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.004771 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.655483 (+650712us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:46.655528 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.655615 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.216077 (+560462us) raft_consensus.cc:1685] finished
0709 13:16:47.216098 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.216224 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":650211,"threads_started":1,"wal-append.queue_time_us":797415}
I20260709 13:16:47.218490 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.219285 11439 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:16:47.224390 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:47.220048 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.136898 16459 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:47.181694 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.228096 11511 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:47.181126 16568 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:47.206512 16578 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:47.206735 16465 consensus_meta.cc:287] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.007s
I20260709 13:16:47.207461 16592 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:47.208351 16575 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:16:47.208751 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2750) took 1202 ms. Trace:
W20260709 13:16:47.209012 16563 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 13:16:47.209311 11438 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
W20260709 13:16:47.210327 16580 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:16:47.141065 16248 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:47.237808 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 152175us
I20260709 13:16:47.117913 16492 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.243517 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:47.212693 16572 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.070s	user 0.006s	sys 0.003s
I20260709 13:16:47.135576 16512 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:47.220012 16570 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:16:47.136809 16479 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.390149 16479 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:47.390601 16564 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:16:47.392177 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1944) took 1052 ms. Trace:
I20260709 13:16:47.392948 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1220) took 1384 ms. Trace:
I20260709 13:16:47.392592 11518 rpcz_store.cc:276] 0709 13:16:46.339580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.340253 (+   673us) service_pool.cc:224] Handling call
0709 13:16:46.340574 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:46.343949 (+  3375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.343974 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:46.395162 (+ 51188us) log.cc:844] Serialized 52 byte log entry
0709 13:16:46.772848 (+377686us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:46.841149 (+ 68301us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.050152 (+209003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.050760 (+   608us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.391959 (+341199us) raft_consensus.cc:1685] finished
0709 13:16:47.391978 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.392122 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5040,"prepare.queue_time_us":68739,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":210679,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":11488,"thread_start_us":485005,"threads_started":2,"wal-append.queue_time_us":492583}
I20260709 13:16:47.262900 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.395541 16467 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.397568 16566 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.398665 16404 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:47.263835 15430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.001s	sys 0.010s
I20260709 13:16:47.262674 11441 rpcz_store.cc:276] 0709 13:16:46.006550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.006685 (+   135us) service_pool.cc:224] Handling call
0709 13:16:47.208536 (+1201851us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002080d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:47.208691 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2381397376}
I20260709 13:16:47.129137 16532 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:47.399226 15430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.001s	sys 0.010s
I20260709 13:16:47.399919 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.178761 16513 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.400535 11441 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:47.201838 15817 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:47.271901 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:47.293538 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:47.295372 11439 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
W20260709 13:16:47.295601 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.383276 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2755) took 1190 ms. Trace:
I20260709 13:16:47.267554 16306 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:47.266921 16592 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.415059 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:47.385993 16460 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:47.387346 16457 consensus_meta.cc:287] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.639s	user 0.007s	sys 0.002s
I20260709 13:16:47.387796 16561 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.223713 16412 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.440338 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.395117 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.441823 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.398903 16036 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.290856 16248 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.401484 11438 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:16:47.435192 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 163369us
I20260709 13:16:47.460165 11441 raft_consensus.cc:2393] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:47.440912 11362 rpcz_store.cc:276] 0709 13:16:46.008562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.008680 (+   118us) service_pool.cc:224] Handling call
0709 13:16:46.008973 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.343727 (+334754us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:46.343757 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.396599 (+ 52842us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.797497 (+400898us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:46.797547 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.797617 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.392744 (+595127us) raft_consensus.cc:1685] finished
0709 13:16:47.392769 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.392896 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8900,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":88,"thread_start_us":635132,"threads_started":2,"wal-append.queue_time_us":494590}
I20260709 13:16:47.458979 11484 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:47.392890 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.435916 11444 rpcz_store.cc:276] 0709 13:16:46.192473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.192727 (+   254us) service_pool.cc:224] Handling call
0709 13:16:46.193120 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.312654 (+119534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.312684 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.328300 (+ 15616us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.732227 (+403927us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b1800475060. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:46.952709 (+220482us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:46.952749 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.953258 (+   509us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.382989 (+429731us) raft_consensus.cc:1685] finished
0709 13:16:47.383012 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.383217 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119162,"raft.queue_time_us":119257,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":7025,"spinlock_wait_cycles":28292480,"thread_start_us":220302,"threads_started":1,"wal-append.queue_time_us":734027}
I20260709 13:16:47.443377 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:47.435429 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 375360us
I20260709 13:16:47.467222 11511 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:16:47.467344 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:47.468030 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:47.468226 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:47.395311 16543 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.897s	user 0.003s	sys 0.000s
I20260709 13:16:47.252595 16568 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:47.456418 16106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.025s	sys 0.027s
I20260709 13:16:47.480846 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:47.473986 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.390184 15885 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.479100 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.265795 16465 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.482952 11444 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:47.456295 16479 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:47.483922 16567 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.464638 16248 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:47.485301 16590 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.485580 12064 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14. Ops: []
W20260709 13:16:47.486012 16586 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.333s	user 0.000s	sys 0.000s
I20260709 13:16:47.487213 16512 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.487597 16513 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.455897 15817 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.488932 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:47.499025 16479 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.490116 15852 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.490258 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:47.490494 16396 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.493073 16460 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.493348 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2757) took 1167 ms. Trace:
I20260709 13:16:47.494179 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:47.483424 16561 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.496575 16595 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:47.484310 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.519898 11514 raft_consensus.cc:2410] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:16:47.483836 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.499370 11438 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:47.499349 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:47.507716 16592 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.500542 11510 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:47.521466 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.522836 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.501138 11516 raft_consensus.cc:2410] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:16:47.297089 16519 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:47.480602 16106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.025s	sys 0.028s
I20260709 13:16:47.474033 16533 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:16:47.497840 16155 consensus_meta.cc:287] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.000s
I20260709 13:16:47.505080 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 2023) took 1387 ms. Trace:
I20260709 13:16:47.490077 16412 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.508780 12067 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:47.496809 16604 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:47.489251 16568 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.462082 16457 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.500110 16562 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:47.507661 16389 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.515239 16479 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:47.524451 11510 raft_consensus.cc:2410] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:16:47.516248 11437 rpcz_store.cc:276] 0709 13:16:46.326172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.489479 (+163307us) service_pool.cc:224] Handling call
0709 13:16:46.513124 (+ 23645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.806566 (+293442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.806597 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.807008 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.049810 (+242802us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:47.049855 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.049922 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.493148 (+443226us) raft_consensus.cc:1685] finished
0709 13:16:47.493170 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.493293 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104336,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8633,"thread_start_us":443809,"threads_started":2,"wal-append.queue_time_us":639596}
I20260709 13:16:47.522681 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:47.528438 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.529415 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:47.526454 11367 rpcz_store.cc:276] 0709 13:16:46.117873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.117983 (+   110us) service_pool.cc:224] Handling call
0709 13:16:46.144481 (+ 26498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.158117 (+ 13636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.158150 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.158629 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.708969 (+550340us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b18001a2900. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:46.942749 (+233780us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:46.942793 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.942916 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.212058 (+269142us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:47.504879 (+292821us) raft_consensus.cc:1685] finished
0709 13:16:47.504900 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.505028 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13253,"raft.queue_time_us":16426,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":120215040,"thread_start_us":233563,"threads_started":1,"wal-append.queue_time_us":908088}
I20260709 13:16:47.498773 16522 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.538203 12032 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:47.540321 11333 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:47.549666 16602 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:47.550321 16534 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:47.551033 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2617) took 1198 ms. Trace:
I20260709 13:16:47.506636 16567 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.554720 16457 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.551891 16595 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:47.556207 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.557096 16483 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.565671 11513 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:16:47.579303 11518 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:47.559340 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.513716 16513 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.533144 16590 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.568490 16460 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:47.584964 16595 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.586488 16567 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.587036 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.587735 11514 raft_consensus.cc:2368] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16.
I20260709 13:16:47.588440 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.589408 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.590345 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.573621 11436 rpcz_store.cc:276] 0709 13:16:46.352674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.636353 (+283679us) service_pool.cc:224] Handling call
0709 13:16:46.636595 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.636789 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:46.640556 (+  3767us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.397248 (+756692us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b18002cd340. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e23a 00007f35bca08a2a
0709 13:16:47.549321 (+152073us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.549394 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:47.549451 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.550660 (+  1209us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.550833 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33664,"prepare.queue_time_us":3995,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":47,"raft.queue_time_us":15842,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":111,"spinlock_wait_cycles":7992192,"thread_start_us":135884,"threads_started":1}
I20260709 13:16:47.575165 12032 leader_election.cc:400] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14.
I20260709 13:16:47.575191 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:47.573817 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 51233.8us
I20260709 13:16:47.577764 16036 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:47.570119 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.610316 11516 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:47.577600 16155 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:47.578091 16492 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:47.577457 16596 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:16:47.591275 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.608371 12032 leader_election.cc:403] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:47.662612 11514 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:47.608491 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:47.663090 11516 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
W20260709 13:16:47.663503 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55118.3us
I20260709 13:16:47.664233 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.611506 11438 raft_consensus.cc:1275] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:47.577843 16389 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:47.542539 16604 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:47.591315 16499 consensus_meta.cc:287] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.001s
W20260709 13:16:47.660027 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44883us
I20260709 13:16:47.567250 16562 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.610247 16532 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:47.667191 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 94725.3us
I20260709 13:16:47.664389 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:47.613528 16483 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.669729 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.615338 16155 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:47.672014 11437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55822
I20260709 13:16:47.668856 16250 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:47.659173 16492 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.709012 11437 raft_consensus.cc:606] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Received request to transfer leadership
I20260709 13:16:47.679520 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:47.576717 16602 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.675829 16306 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:47.592321 11254 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6e87b213a4554d94825aceec352fc094 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:47.710287 11437 raft_consensus.cc:614] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:47.608824 16460 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:47.725997 11254 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 133836us
W20260709 13:16:47.711524 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47235us
I20260709 13:16:47.664920 16568 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.711990 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:47.713289 16606 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:16:47.713888 16604 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.715549 16562 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.716075 16250 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.666380 16036 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.670974 16414 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:47.709968 11516 raft_consensus.cc:2410] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:16:47.717021 16492 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:47.721511 12032 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:47.727370 11514 raft_consensus.cc:2376] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11. Re-sending same reply.
I20260709 13:16:47.727694 11333 leader_election.cc:400] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16.
I20260709 13:16:47.739450 11333 leader_election.cc:403] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:16:47.676640 16596 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:47.679634 16499 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.718442 16155 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:47.710007 16465 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:16:47.717059 16532 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:47.719596 16600 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.677307 16389 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.725338 16371 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.681623 16411 consensus_queue.cc:1048] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:16:47.741138 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:47.773104 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:47.773500 16554 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.774024 16554 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.776288 16370 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.777221 16389 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:47.780686 11445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51484 (request call id 2614) took 1432 ms (client timeout 1838 ms). Trace:
I20260709 13:16:47.781008 11443 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:16:47.781102 11445 rpcz_store.cc:269] 0709 13:16:46.348684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.368065 (+ 19381us) service_pool.cc:224] Handling call
0709 13:16:47.780508 (+1412443us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003e49d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:47.780628 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360062848}
W20260709 13:16:47.779815 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.742213 11516 raft_consensus.cc:1240] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:47.730831 16306 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.732118 16568 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.743058 16036 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:47.743732 11333 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:47.789343 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 45693.6us
I20260709 13:16:47.789973 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:47.790452 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:47.732609 15817 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:47.748185 16436 consensus_meta.cc:287] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 1.054s	user 0.010s	sys 0.001s
I20260709 13:16:47.752430 16373 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:47.754220 16612 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:16:47.754671 16584 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.755874 16602 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.680493 16412 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.820326 16512 consensus_queue.cc:1059] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:47.820739 16306 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:47.821022 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.809698 16515 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.821022 16596 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.787403 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.790695 12066 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:47.822103 16515 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.823948 16460 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.824420 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.825086 11366 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:47.825953 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:47.826449 11366 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:47.744530 16250 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.827656 16513 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.796309 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.828351 16436 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.828392 11369 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:47.829178 11514 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:16:47.828903 16436 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.831009 16436 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:47.762966 15852 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:47.832187 15852 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:47.833127 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.834232 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:47.793601 16328 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:47.836737 16585 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:47.837414 16512 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:47.819979 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.837841 16512 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:47.838981 16512 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.840013 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.840867 11369 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:47.837350 16585 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.841786 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.842911 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.802064 11361 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:47.844215 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.845381 11361 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:47.776182 16554 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:47.807206 16389 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.803243 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:47.851853 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.811367 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:47.812723 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:47.839110 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.852725 12064 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:47.770771 16600 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.815654 16483 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.818094 16568 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:47.819005 16614 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:16:47.820106 16412 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.822003 11365 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:47.823132 16241 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:47.788754 16465 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:47.852167 16579 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.598s	user 0.004s	sys 0.000s
I20260709 13:16:47.852695 11511 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:47.814656 16584 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:47.784358 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:47.794806 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.871913 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2583) took 1491 ms. Trace:
I20260709 13:16:47.854905 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.021700 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2626) took 1062 ms. Trace:
I20260709 13:16:48.045712 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:47.870280 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.046186 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.916517 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.073484 11516 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
W20260709 13:16:48.020686 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 214342us
I20260709 13:16:48.027287 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:47.932421 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:48.077522 11361 raft_consensus.cc:1813] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Handling vote request from an unknown peer 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:48.043952 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:48.078727 11361 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:16:48.054529 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1951) took 1269 ms. Trace:
I20260709 13:16:48.069240 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:47.856132 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.079044 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:48.076017 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:48.079250 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 294954us
I20260709 13:16:48.081568 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:48.082314 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:48.083688 16250 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:48.075736 11510 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:48.044927 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2760) took 1283 ms. Trace:
W20260709 13:16:47.871392 16577 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.725s	user 0.000s	sys 0.001s
W20260709 13:16:47.856398 16576 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.779s	user 0.001s	sys 0.001s
I20260709 13:16:48.076630 11437 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:47.843209 16585 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:47.858335 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:48.106585 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 248323us
I20260709 13:16:47.834224 15852 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:47.860289 16571 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:16:47.877440 16513 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:48.085668 11512 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:48.051183 11268 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 8 to 10, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 10 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:48.085281 16594 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.114995 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.115693 11518 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:48.091022 16554 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:48.035347 11366 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:48.088994 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.124656 11442 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:16:48.125569 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.085955 11334 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:48.088310 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1225) took 1892 ms. Trace:
I20260709 13:16:48.041715 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.090762 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1102) took 1031 ms. Trace:
I20260709 13:16:47.840293 16512 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.088168 11443 rpcz_store.cc:276] 0709 13:16:46.761811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.762599 (+   788us) service_pool.cc:224] Handling call
0709 13:16:47.780649 (+1018050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.781578 (+   929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.044587 (+263009us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003e49d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:48.044841 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340624256}
I20260709 13:16:48.128715 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:48.092378 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.088723 11439 rpcz_store.cc:276] 0709 13:16:46.959821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.437090 (+477269us) service_pool.cc:224] Handling call
0709 13:16:47.437464 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.437681 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.437706 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.440788 (+  3082us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.540252 (+ 99464us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:47.540300 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.540370 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.021480 (+481110us) raft_consensus.cc:1685] finished
0709 13:16:48.021500 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.021641 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":51242,"threads_started":1,"wal-append.queue_time_us":131503}
I20260709 13:16:48.079806 11519 rpcz_store.cc:276] 0709 13:16:46.788123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.802292 (+ 14169us) service_pool.cc:224] Handling call
0709 13:16:46.845804 (+ 43512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.084625 (+238821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.084681 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.118221 (+ 33540us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.497613 (+379392us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:47.497653 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.858735 (+361082us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540014ffd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:47.858784 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.858832 (+    48us) raft_consensus.cc:1685] finished
0709 13:16:47.858845 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.054459 (+195614us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143541,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":8042,"spinlock_wait_cycles":349784960,"thread_start_us":429451,"threads_started":2,"wal-append.queue_time_us":427213}
I20260709 13:16:48.093942 11408 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.132381 11519 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:47.870244 16248 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.073722 11370 rpcz_store.cc:276] 0709 13:16:46.380725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.385991 (+  5266us) service_pool.cc:224] Handling call
0709 13:16:47.871720 (+1485729us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540035b954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:47.871833 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837904128}
I20260709 13:16:48.098678 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:48.022707 16414 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14.
I20260709 13:16:47.944629 16534 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.867655 16499 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.141259 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2585) took 1442 ms. Trace:
I20260709 13:16:47.867756 16535 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:16:48.145390 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 59511.5us
I20260709 13:16:48.078886 11438 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.108219 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.144759 11368 rpcz_store.cc:276] 0709 13:16:46.698686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.698847 (+   161us) service_pool.cc:224] Handling call
0709 13:16:46.699255 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.699657 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.699693 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.703465 (+  3772us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.743963 (+ 40498us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b180018fdc0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:46.938716 (+194753us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:46.938758 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.938827 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.829359 (+890532us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:48.141040 (+311681us) raft_consensus.cc:1685] finished
0709 13:16:48.141067 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.141204 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3430,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":2878,"spinlock_wait_cycles":17205760,"thread_start_us":194550,"threads_started":1,"wal-append.queue_time_us":346427}
I20260709 13:16:48.107205 11361 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:48.102615 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.102383 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.028187 16389 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:48.048323 16515 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:47.860939 16467 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:48.050486 11266 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39438 (request call id 361) took 15349 ms (client timeout 15000 ms). Trace:
I20260709 13:16:48.149448 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:48.111339 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:47.865685 16522 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.032356 16465 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:48.082571 11367 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:48.126430 11511 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:48.126681 11513 rpcz_store.cc:276] 0709 13:16:47.060578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.060821 (+   243us) service_pool.cc:224] Handling call
0709 13:16:47.565350 (+504529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.578971 (+ 13621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.090579 (+511608us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54000645d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:48.090706 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146914688}
I20260709 13:16:48.100174 16328 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.100350 16377 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:48.127673 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.174731 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1224) took 2129 ms. Trace:
I20260709 13:16:48.174803 11365 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:48.175854 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.176448 11365 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
W20260709 13:16:48.177491 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 66225.8us
I20260709 13:16:48.132077 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:48.072384 16483 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:48.077533 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:48.126395 11364 rpcz_store.cc:276] 0709 13:16:46.195466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.197194 (+  1728us) service_pool.cc:224] Handling call
0709 13:16:46.197472 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.416742 (+219270us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:46.496504 (+ 79762us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.838258 (+341754us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b18000c7c00. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e23a 00007f35bca08a2a
0709 13:16:46.975058 (+136800us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.975115 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:46.975165 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.088109 (+1112944us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540024a754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:48.088155 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.088261 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1964,"prepare.queue_time_us":79601,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":107,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":5853,"spinlock_wait_cycles":516699264,"thread_start_us":136533,"threads_started":1,"wal-append.queue_time_us":570757}
I20260709 13:16:48.133008 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.183233 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.183995 11439 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:48.131907 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:48.126878 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.114033 16397 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:48.112353 15817 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:48.139276 11519 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:48.129341 11443 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:48.141541 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.148057 16535 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:48.192164 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.191764 11268 catalog_manager.cc:5697] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 7 to 9, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 9 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:48.142892 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.144028 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.149694 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.151360 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2627) took 1066 ms. Trace:
I20260709 13:16:48.154964 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:48.135998 16479 consensus_meta.cc:287] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.001s
I20260709 13:16:48.147889 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.122078 16594 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.197981 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:48.136482 16569 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:48.170411 11484 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:48.189321 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.198957 12069 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:48.199623 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:48.166884 11440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2759) took 1758 ms. Trace:
I20260709 13:16:48.200014 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.195251 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:48.195514 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.202672 12063 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:48.196597 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.184870 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:48.186149 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54316.1us
I20260709 13:16:48.186510 11366 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:48.111025 16571 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:48.175213 11363 rpcz_store.cc:276] 0709 13:16:46.045659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.062119 (+ 16460us) service_pool.cc:224] Handling call
0709 13:16:46.062417 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:46.642688 (+580271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.642717 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:46.718894 (+ 76177us) log.cc:844] Serialized 52 byte log entry
0709 13:16:46.994021 (+275127us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:46.994065 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.818375 (+824310us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:47.840815 (+ 22440us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b54000c6554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:47.840871 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.174488 (+333617us) raft_consensus.cc:1685] finished
0709 13:16:48.174510 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.174673 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85287,"raft.queue_time_us":98921,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"spinlock_wait_cycles":139061888,"thread_start_us":663078,"threads_started":2,"wal-append.queue_time_us":348029}
W20260709 13:16:48.186661 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 53730us
I20260709 13:16:48.187947 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.187108 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.190698 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.187201 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:48.119526 16513 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:48.155030 16532 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:48.158728 11266 rpcz_store.cc:269] 0709 13:16:32.700980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.702244 (+  1264us) service_pool.cc:224] Handling call
0709 13:16:48.050404 (+15348160us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:43.256585 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:43.257193 (+   608us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:43.257243 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:16:43.257259 (+    16us) tablet.cc:662] Decoding operations
0709 13:16:43.258712 (+  1453us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:43.258819 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:43.258844 (+    25us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:16:43.296217 (+ 37373us) tablet.cc:701] Row locks acquired
0709 13:16:43.296264 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:16:43.296442 (+   178us) write_op.cc:270] Start()
0709 13:16:43.296592 (+   150us) write_op.cc:276] Timestamp: P: 1783603003296379 usec, L: 0
0709 13:16:43.296631 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:16:43.297341 (+   710us) log.cc:844] Serialized 7474 byte log entry
0709 13:16:43.750046 (+452705us) op_driver.cc:464] REPLICATION: finished
0709 13:16:43.942815 (+192769us) write_op.cc:301] APPLY: starting
0709 13:16:43.977486 (+ 34671us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:44.199837 (+222351us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:44.199966 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:44.777182 (+577216us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:44.777552 (+   370us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:16:44.777612 (+    60us) write_op.cc:312] APPLY: finished
0709 13:16:44.778042 (+   430us) log.cc:844] Serialized 172 byte log entry
0709 13:16:44.809769 (+ 31727us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:44.838907 (+ 29138us) write_op.cc:454] Released schema lock
0709 13:16:44.857654 (+ 18747us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:44.861769 (+  4115us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:44.861803 (+    34us) write_op.cc:454] Released schema lock
0709 13:16:45.766767 (+904964us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
Metrics: {"mutex_wait_us":1228500,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":192505,"apply.run_cpu_time_us":35701,"apply.run_wall_time_us":915283,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83471,"num_ops":19,"prepare.queue_time_us":93255,"prepare.run_cpu_time_us":20738,"prepare.run_wall_time_us":76093,"raft.queue_time_us":227875,"raft.run_cpu_time_us":61301,"raft.run_wall_time_us":228636,"replication_time_us":453134,"spinlock_wait_cycles":4165504,"thread_start_us":397857,"threads_started":5,"wal-append.queue_time_us":136984,"wal-append.run_cpu_time_us":37032,"wal-append.run_wall_time_us":3153850}]]}
I20260709 13:16:48.197273 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.197726 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.135759 16554 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:48.192556 16596 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:48.269831 16596 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:48.270602 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.194746 11361 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:48.194628 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.271857 11437 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:48.200611 12061 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:48.272810 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.270243 16596 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:48.199065 11519 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:48.274170 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.168560 16465 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:48.274724 11519 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:16:48.164078 16036 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16.
W20260709 13:16:48.275655 16620 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.175s	user 0.003s	sys 0.000s
I20260709 13:16:48.276546 15852 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:48.277225 11437 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
W20260709 13:16:48.277711 16618 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
I20260709 13:16:48.275626 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:48.196844 16599 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:48.280644 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:48.281631 11519 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.282179 16571 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.283658 16571 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:48.272107 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:48.285384 11361 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.273082 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.289368 11514 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:48.197852 16414 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:16:48.291646 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1106) took 1224 ms. Trace:
I20260709 13:16:48.291991 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.292905 12061 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:48.205706 16499 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:48.196885 15953 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:48.293345 16499 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:48.293867 16499 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:48.295287 11365 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:48.296955 15817 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:48.297350 15817 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.297571 11365 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:16:48.297665 15817 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:48.165988 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.153585 11510 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
W20260709 13:16:48.261986 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 91658.1us
I20260709 13:16:48.295326 16604 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:48.299690 16531 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:48.300040 16604 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.300863 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.293650 15953 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:48.301709 15953 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.302114 16604 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.206378 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.302664 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.303733 16336 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:48.304100 16336 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.262460 11443 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:48.304428 16336 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:48.304617 16519 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:48.306021 16519 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:48.306358 16519 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.306708 16519 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:48.307996 11510 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.308359 11510 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:48.168851 16460 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:48.263665 11370 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:48.305069 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.309581 11443 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:48.310082 16568 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:48.310489 11370 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:48.310609 16568 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:48.292063 11517 rpcz_store.cc:276] 0709 13:16:47.067142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.067543 (+   401us) service_pool.cc:224] Handling call
0709 13:16:47.080400 (+ 12857us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:47.269178 (+188778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.271789 (+  2611us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:47.777657 (+505868us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b2c000fe208. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:47.778263 (+   606us) log.cc:844] Serialized 52 byte log entry
0709 13:16:47.792818 (+ 14555us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:47.836256 (+ 43438us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.860548 (+ 24292us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.860778 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.860858 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.275099 (+414241us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:48.291433 (+ 16334us) raft_consensus.cc:1685] finished
0709 13:16:48.291454 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.291589 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":151455,"prepare.queue_time_us":74024,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":138981,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":47605,"spinlock_wait_cycles":18755072,"thread_start_us":162489,"threads_started":2,"wal-append.queue_time_us":185893}
I20260709 13:16:48.264357 16569 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:48.316609 16569 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:48.318734 16248 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.212997 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.322638 11362 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 9: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:48.323117 12063 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:48.264109 16520 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:48.324438 16520 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:48.324853 16520 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:48.325191 16336 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.325629 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.192481 16522 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 13:16:48.327245 16410 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:48.327641 16410 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:48.328786 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.328471 16410 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.329708 16410 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:48.331141 16617 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:16:48.263474 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.174288 16585 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.269260 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.270277 16554 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.193646 11518 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:48.206223 16479 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.312652 16568 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:48.356562 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:48.263913 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.331789 16592 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:48.331653 16569 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:48.200186 11440 rpcz_store.cc:276] 0709 13:16:46.408666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.644683 (+236017us) service_pool.cc:224] Handling call
0709 13:16:46.644977 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.693600 (+ 48623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.693632 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.694136 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.943012 (+248876us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:46.943071 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.943157 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.870643 (+927486us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:48.158339 (+287696us) raft_consensus.cc:1685] finished
0709 13:16:48.158382 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.166820 (+  8438us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57599,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":775,"spinlock_wait_cycles":6089600,"thread_start_us":178975,"threads_started":1,"wal-append.queue_time_us":427414}
I20260709 13:16:48.158084 16389 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.333294 16599 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:48.335165 16535 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.148622 16534 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:48.336198 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.336617 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.264791 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.140754 16492 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.194018 11444 rpcz_store.cc:276] 0709 13:16:47.085360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.501215 (+415855us) service_pool.cc:224] Handling call
0709 13:16:47.501551 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.501959 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:47.501987 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.506080 (+  4093us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.613884 (+107804us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:47.613925 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.613992 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.151168 (+537176us) raft_consensus.cc:1685] finished
0709 13:16:48.151188 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.151306 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":8946,"thread_start_us":95537,"threads_started":1,"wal-append.queue_time_us":213030}
I20260709 13:16:48.340771 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:48.266709 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.344365 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.363667 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.366235 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.302050 16584 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.382508 11366 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:48.180744 16328 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.349215 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.349516 12066 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:16:48.349740 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:48.350178 11409 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.020s	sys 0.053s
W20260709 13:16:48.452616 11409 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.021s	sys 0.054s
I20260709 13:16:48.360934 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:48.345422 16336 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.455118 11364 raft_consensus.cc:2368] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:48.378113 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:48.329226 16616 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:16:48.293473 16412 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:48.393229 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 128640us
I20260709 13:16:48.292016 16600 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.458099 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:48.204823 12067 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:16:48.451519 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 110833us
I20260709 13:16:48.451753 11518 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:48.496491 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:48.497387 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.385682 16599 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.207144 11516 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:16:48.455469 12066 pending_rounds.cc:85] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 11
I20260709 13:16:48.458943 11442 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:48.500824 11515 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1110) took 1419 ms. Trace:
W20260709 13:16:48.455996 16278 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.024s	sys 0.061s
I20260709 13:16:48.395342 16568 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:48.508153 16515 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:48.458187 16569 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:16:48.466447 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 116774us
I20260709 13:16:48.513592 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:48.467082 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 75167.4us
I20260709 13:16:48.470079 11361 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:48.470907 15953 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:48.472472 11369 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
W20260709 13:16:48.476475 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 127230us
I20260709 13:16:48.466609 16521 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:48.519042 16521 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:48.519451 16521 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:48.519809 15953 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.524804 16594 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.526335 16594 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:48.528671 16479 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.529522 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2546) took 1011 ms. Trace:
I20260709 13:16:48.530297 16492 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.350337 16531 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.477622 11510 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:48.506290 16278 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.024s	sys 0.061s
W20260709 13:16:48.478889 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 110977us
I20260709 13:16:48.498479 16567 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.529896 11514 rpcz_store.cc:276] 0709 13:16:47.518248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.290094 (+771846us) service_pool.cc:224] Handling call
0709 13:16:48.290357 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.342764 (+ 52407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.342797 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.342835 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:48.342875 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.529311 (+186436us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b54000668d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:48.529352 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.529464 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120056192}
W20260709 13:16:48.545440 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.551219 11511 raft_consensus.cc:2393] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 11: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:48.483171 16520 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.488377 12069 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:48.489061 12062 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:48.490144 12064 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
W20260709 13:16:48.483127 16155 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.000s
I20260709 13:16:48.348429 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:48.570680 11443 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:48.362752 16592 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.502875 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:48.503291 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:48.501204 11515 rpcz_store.cc:276] 0709 13:16:47.081586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.730677 (+649091us) service_pool.cc:224] Handling call
0709 13:16:47.730975 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.731257 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.731275 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.731696 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 13:16:48.086134 (+354438us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:48.086174 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.086237 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.500606 (+414369us) raft_consensus.cc:1685] finished
0709 13:16:48.500627 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.500764 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1239,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":1619,"spinlock_wait_cycles":1956608,"thread_start_us":351606,"threads_started":1,"wal-append.queue_time_us":401655}
I20260709 13:16:48.502355 11362 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:48.500154 12066 pending_rounds.cc:107] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:16:48.572430 12066 raft_consensus.cc:2889] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:48.465112 16596 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:48.574432 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 420526us
I20260709 13:16:48.355505 16585 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:48.465621 11519 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:48.517751 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:48.495448 16535 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:48.518517 16600 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:48.478871 11438 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
W20260709 13:16:48.603356 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 85656.1us
I20260709 13:16:48.535223 11512 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:48.526403 11444 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:16:48.455026 16604 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:48.561050 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.576750 16569 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:48.566272 11510 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.567996 16483 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.568799 12033 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:48.569217 16616 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:16:48.569576 16531 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.506700 16568 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:48.573844 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70645us
I20260709 13:16:48.633687 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:48.634402 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:48.578570 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:48.635082 11515 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:48.581888 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.635969 11518 raft_consensus.cc:2410] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:48.602146 12065 raft_consensus.cc:1813] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Handling vote request from an unknown peer 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:48.602237 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.564630 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:48.636665 12065 raft_consensus.cc:2410] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:16:48.636775 11442 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:48.612972 11440 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:48.612864 12060 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:16:48.573499 16592 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:48.623085 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 473687us
I20260709 13:16:48.610939 16092 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:48.595329 16457 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:48.621781 11439 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:48.630932 16629 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:48.647645 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:48.648255 11439 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:48.653765 11442 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:48.666990 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:48.655639 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 86904.1us
I20260709 13:16:48.668509 12033 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:48.658448 11439 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:48.659317 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:36660 (request call id 2534) took 1133 ms. Trace:
I20260709 13:16:48.669415 11510 rpcz_store.cc:276] 0709 13:16:47.485825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.153468 (+667643us) service_pool.cc:224] Handling call
0709 13:16:48.618784 (+465316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.659789 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:48.670948 11440 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:48.660267 11445 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.006s	sys 0.033s
I20260709 13:16:48.626971 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:48.661939 11361 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:48.662911 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.602008 16578 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:48.601536 16510 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:48.681344 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 54472.3us
I20260709 13:16:48.664996 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:48.602900 16600 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:48.679754 11515 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:48.681484 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:48.684953 11444 raft_consensus.cc:2410] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:16:48.574437 16567 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.594816 16502 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:16:48.663861 11443 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 18.
W20260709 13:16:48.674443 11445 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.006s	sys 0.033s
I20260709 13:16:48.675256 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:48.691836 12033 leader_election.cc:400] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:48.694931 12033 leader_election.cc:403] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:48.682507 11367 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:48.695128 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.682432 11364 raft_consensus.cc:2393] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:16:48.683081 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48707.7us
I20260709 13:16:48.698233 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:48.576856 16155 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:48.687702 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 194638us
I20260709 13:16:48.615228 16336 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:48.691955 11334 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.009s	sys 0.035s
I20260709 13:16:48.695752 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:48.705331 11334 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.010s	sys 0.035s
I20260709 13:16:48.633612 16604 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:48.621054 16278 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
W20260709 13:16:48.706225 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 559838us
I20260709 13:16:48.707105 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.692358 16569 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:16:48.615602 16532 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:16:48.673593 16530 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:48.711200 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:48.719503 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 107258us
I20260709 13:16:48.680649 16594 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:48.715822 12068 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:48.717988 11334 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:48.692474 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 234430us
I20260709 13:16:48.687508 16492 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:48.686673 16600 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:48.691731 16522 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:48.688056 16584 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:48.752115 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:48.704910 16631 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:48.874491 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.708593 16306 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:48.709816 16328 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.756465 16594 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.896242 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2628) took 1810 ms. Trace:
I20260709 13:16:48.723582 16479 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:48.687893 16578 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:48.909523 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 157505us
I20260709 13:16:48.911562 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2592) took 1062 ms. Trace:
I20260709 13:16:48.911814 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2550) took 1113 ms. Trace:
I20260709 13:16:48.914038 11441 rpcz_store.cc:276] 0709 13:16:47.085611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.536839 (+451228us) service_pool.cc:224] Handling call
0709 13:16:47.537171 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.617904 (+ 80733us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:47.677540 (+ 59636us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.102610 (+425070us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:48.103138 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:16:48.103263 (+   125us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:48.103295 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.287939 (+184644us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b5400398854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:48.287995 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.757300 (+469305us) raft_consensus.cc:1685] finished
0709 13:16:48.896041 (+138741us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.896183 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18225,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":3812,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":50108,"spinlock_wait_cycles":176340864,"thread_start_us":377893,"threads_started":1,"wal-append.queue_time_us":433186}
I20260709 13:16:48.874993 16492 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.031917 16492 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:49.035604 16457 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.036137 16457 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.038256 16457 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:48.687098 16510 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.873636 16250 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:49.040936 16510 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.041491 16510 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:48.755075 16410 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.165s
I20260709 13:16:48.895032 16534 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:48.909305 16602 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:49.045087 16602 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:49.045138 16519 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.738s	user 0.007s	sys 0.000s
I20260709 13:16:49.045749 16519 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.046105 16602 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.047784 16519 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:48.970273 16577 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:16:48.757007 16245 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.215s
W20260709 13:16:48.971110 16626 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 13:16:48.914382 16631 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:48.922147 16594 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:48.870241 16499 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:48.972663 16389 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.082568 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:48.914264 16522 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:48.719913 16590 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:48.763087 16467 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:49.025169 11512 rpcz_store.cc:276] 0709 13:16:47.798005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.320067 (+522062us) service_pool.cc:224] Handling call
0709 13:16:48.320349 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.604405 (+284056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.911596 (+307191us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540014c3d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:48.911760 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493779328}
I20260709 13:16:49.025815 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:49.025100 11369 rpcz_store.cc:276] 0709 13:16:47.849171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.932087 (+ 82916us) service_pool.cc:224] Handling call
0709 13:16:48.911379 (+979292us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54001e5ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:48.911497 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462535936}
W20260709 13:16:49.027907 16624 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.406s	user 0.000s	sys 0.001s
I20260709 13:16:48.897526 16584 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:49.054483 15817 consensus_meta.cc:287] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.756s	user 0.006s	sys 0.002s
W20260709 13:16:49.061817 16397 consensus_meta.cc:287] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.870s	user 0.007s	sys 0.000s
I20260709 13:16:49.067891 16585 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.084307 11519 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:49.105141 12061 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
W20260709 13:16:49.108516 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 398238us
I20260709 13:16:48.905015 16460 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.006916 16513 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.023514 16278 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:49.077339 16634 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:16:49.093597 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:49.000567 16604 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:49.096639 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 70881.9us
I20260709 13:16:49.086542 16631 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:49.122776 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.123488 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.067883 16592 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.124343 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:48.986222 16306 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.107568 16519 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:49.104473 16397 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.089248 16155 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.095434 16584 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:49.123756 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.157087 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.179476 11512 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:49.094347 16467 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.158972 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:49.140833 11440 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:49.102273 16569 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:49.102566 16520 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:49.162936 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 481815us
W20260709 13:16:49.157406 16512 consensus_meta.cc:287] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.019s	user 0.009s	sys 0.000s
W20260709 13:16:49.156823 16465 consensus_meta.cc:287] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.882s	user 0.006s	sys 0.003s
I20260709 13:16:49.177318 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1257) took 1407 ms. Trace:
W20260709 13:16:49.149219 16412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.056s	sys 0.144s
I20260709 13:16:49.095762 15817 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.196090 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:49.162035 16590 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:16:49.116284 11410 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.198490 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:16:49.200811 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 482887us
I20260709 13:16:49.146533 16460 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:49.201579 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 368887us
I20260709 13:16:49.203066 12063 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:49.203540 12068 rpcz_store.cc:276] 0709 13:16:47.769494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.769619 (+   125us) service_pool.cc:224] Handling call
0709 13:16:47.769910 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.756832 (+986922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.177082 (+420250us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54001faed4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:49.177247 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175014784}
I20260709 13:16:49.205536 11369 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:49.213660 11369 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:49.170717 16513 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:49.210682 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:49.211987 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:49.216998 11440 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:49.214424 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.171219 16595 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:16:49.214426 11369 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:49.206171 16412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.056s	sys 0.144s
I20260709 13:16:49.225980 11365 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:49.222640 11369 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:49.199018 16467 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:49.231854 12032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.014s	sys 0.019s
W20260709 13:16:49.232193 12032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.014s	sys 0.019s
I20260709 13:16:49.233307 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.233800 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.234987 11335 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:49.235111 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:49.235720 12067 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:49.236266 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:49.236471 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:49.236997 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:49.237577 12069 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:49.243640 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:49.246272 12066 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:16:49.246191 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.247155 12060 raft_consensus.cc:2410] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:16:49.247630 12064 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:49.248067 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 497532us
I20260709 13:16:49.218349 16596 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.249022 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:49.227258 16590 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.266147 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:49.268500 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.227617 11512 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:16:49.206447 15852 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:49.229485 12034 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.012s	sys 0.011s
I20260709 13:16:49.208076 16465 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:49.273663 12034 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.012s	sys 0.012s
I20260709 13:16:49.248076 16534 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:49.215750 16519 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:16:49.222004 16513 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.276141 12067 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 18.
I20260709 13:16:49.227855 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.281710 12064 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:49.278316 12069 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:49.288897 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.286985 11369 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
W20260709 13:16:49.290637 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 583591us
I20260709 13:16:49.263620 16554 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:16:49.264788 16412 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:16:49.317008 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 95935.5us
I20260709 13:16:49.325435 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.317963 16630 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.262161 16650 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:49.317066 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 104939us
I20260709 13:16:49.344868 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:49.339155 11364 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:16:49.339488 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:49.347573 11361 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 18.
I20260709 13:16:49.340186 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:49.290067 16653 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:49.344791 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:49.346998 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.340835 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:49.333927 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:49.351552 11441 raft_consensus.cc:2368] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 17. Current term is 18.
I20260709 13:16:49.290552 16647 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:49.350353 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:49.350415 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:49.354354 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 119442us
I20260709 13:16:49.359895 11335 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:49.361676 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:49.362574 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:49.363509 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.364508 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.365530 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.366257 12069 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 8: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:49.367554 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.342469 16412 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.331318 16499 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.325126 16578 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.348896 16092 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:49.232185 16522 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.370221 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.376683 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:49.379904 11366 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:49.359493 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.348209 16654 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:49.357884 16602 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:49.368716 15852 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:49.388576 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48653.2us
I20260709 13:16:49.370438 11441 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:49.373479 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:49.387475 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:49.374876 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 125833us
I20260709 13:16:49.390396 11367 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:49.369887 16389 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:16:49.370092 16592 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.349972 16650 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:49.386365 16639 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:16:49.367890 16250 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:49.361186 16653 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:49.389441 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:49.400472 12034 leader_election.cc:400] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 17. Current term is 18.
I20260709 13:16:49.370138 16655 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:49.382153 16642 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 0.188s	user 0.002s	sys 0.000s
I20260709 13:16:49.395217 16595 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:49.416421 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.300740 16534 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:49.390956 11333 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:49.405999 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 139917us
I20260709 13:16:49.381906 16654 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.403941 16648 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:16:49.357949 16590 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.387495 15953 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:49.410578 12034 leader_election.cc:403] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:16:49.388191 16554 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:49.377306 16578 consensus_queue.cc:1059] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:16:49.405596 16569 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.382465 16652 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.394395 16389 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:49.483894 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 93006.9us
W20260709 13:16:49.410971 16644 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.207s	user 0.001s	sys 0.001s
I20260709 13:16:49.404346 15817 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.422708 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:49.403053 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 43649.1us
I20260709 13:16:49.480108 12064 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:49.490762 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:49.406735 16650 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:49.382223 16092 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:49.487430 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 90156.5us
I20260709 13:16:49.397406 16519 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
W20260709 13:16:49.420120 16641 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:16:49.419227 16534 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:49.492799 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:49.493222 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 75506.7us
W20260709 13:16:49.488168 16639 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:16:49.497099 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 74484us
I20260709 13:16:49.519985 11409 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:49.521261 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.372082 16412 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.480278 16656 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:49.530508 11364 raft_consensus.cc:1240] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:16:49.538998 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:49.492210 16155 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.498869 15953 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
W20260709 13:16:49.499511 16657 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:16:49.500770 16092 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:49.496599 16602 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:16:49.501348 16658 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:16:49.565979 16646 log.cc:927] Time spent T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:16:49.501529 16660 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:49.566557 16660 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:49.507699 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2783) took 1177 ms. Trace:
I20260709 13:16:49.417398 16655 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.567371 11436 rpcz_store.cc:276] 0709 13:16:48.330774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.507425 (+176651us) service_pool.cc:224] Handling call
0709 13:16:48.552574 (+ 45149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:48.572508 (+ 19934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.572538 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:48.572984 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:16:48.897744 (+324760us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:48.897783 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.897846 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:49.507489 (+609643us) raft_consensus.cc:1685] finished
0709 13:16:49.507511 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.507639 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25894,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7472,"thread_start_us":278389,"threads_started":1,"wal-append.queue_time_us":596420}
I20260709 13:16:49.513758 16650 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.488274 16389 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:16:49.500604 16630 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:49.421852 16595 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:49.497871 16643 log.cc:927] Time spent T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:16:49.416721 16664 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:16:49.532126 16585 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:16:49.533088 16653 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.575584 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.541858 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.496639 16648 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.492076 16645 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 17. Current term is 18.
I20260709 13:16:49.487428 16662 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:49.548535 15852 consensus_queue.cc:1059] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:16:49.484105 16554 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.577450 16652 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:49.601763 16652 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.602751 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.603607 12069 raft_consensus.cc:2410] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:49.603977 16652 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:49.530817 16521 consensus_meta.cc:287] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 1.011s	user 0.010s	sys 0.000s
W20260709 13:16:49.604406 16492 consensus_meta.cc:287] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.000s
I20260709 13:16:49.495781 16465 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.604820 16567 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.600741 16568 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.532998 16569 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.606366 16568 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:49.576731 16604 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:16:49.532677 16535 consensus_meta.cc:287] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.930s	user 0.009s	sys 0.000s
I20260709 13:16:49.565207 16602 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:49.541350 16414 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.624987 16414 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.626101 11367 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:49.626457 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.538494 16512 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:49.627573 16512 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.531803 16656 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.628072 16457 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:49.604833 16492 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.628909 16656 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:49.629842 16625 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.125s	user 0.012s	sys 0.000s
I20260709 13:16:49.631057 16492 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.547089 16651 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:49.583760 16621 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.005s	user 0.009s	sys 0.003s
I20260709 13:16:49.568274 16513 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.545706 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:49.515385 16600 consensus_meta.cc:287] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.000s
I20260709 13:16:49.548880 16155 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.599776 16519 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.648408 16651 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.651506 16155 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:49.630965 16245 raft_consensus.cc:3055] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:49.600687 16595 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.601225 16592 raft_consensus.cc:3055] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:49.547765 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 53422.4us
I20260709 13:16:49.601125 16584 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.661386 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:49.649911 16519 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.515926 16306 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.423564 16654 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:49.610270 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.575019 16630 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:49.585300 16631 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.619871 16602 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:49.619593 16521 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.620465 16535 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:49.687599 16535 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:49.688390 16535 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.688865 16521 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.689498 16535 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:49.621883 16567 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:49.623788 11438 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:16:49.632757 16554 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.692902 11438 pending_rounds.cc:85] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 11
I20260709 13:16:49.693204 11438 pending_rounds.cc:107] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:16:49.693467 11438 raft_consensus.cc:2889] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:49.615674 16662 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:49.634788 16467 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.593029 16648 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.638048 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.581475 16649 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:49.567013 16660 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:49.627138 16414 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.627624 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:49.607656 16599 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 1.095s	user 0.010s	sys 0.001s
I20260709 13:16:49.645669 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:49.628010 16512 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:49.625895 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2773) took 1530 ms. Trace:
W20260709 13:16:49.641964 16594 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.004s
I20260709 13:16:49.649209 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.708787 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.709082 12032 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:49.652204 16245 raft_consensus.cc:740] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.645695 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:49.711843 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 66248us
I20260709 13:16:49.705298 11437 rpcz_store.cc:276] 0709 13:16:48.095456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.278012 (+182556us) service_pool.cc:224] Handling call
0709 13:16:48.278328 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.513810 (+235482us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:48.558236 (+ 44426us) log.cc:844] Serialized 14 byte log entry
0709 13:16:49.217894 (+659658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:49.217942 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:49.217989 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.625644 (+407655us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540027a1d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:49.625700 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.625840 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50057,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":290140,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":541055744,"thread_start_us":659249,"threads_started":2,"wal-append.queue_time_us":425409}
I20260709 13:16:49.714308 11518 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:49.649827 16600 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.654875 16592 raft_consensus.cc:740] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.659047 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2556) took 1074 ms. Trace:
I20260709 13:16:49.691167 16651 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.718238 11519 rpcz_store.cc:276] 0709 13:16:48.584195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.601974 (+ 17779us) service_pool.cc:224] Handling call
0709 13:16:49.658877 (+1056903us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54002a80d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:49.658986 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029453056}
I20260709 13:16:49.662158 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:49.661605 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:49.720760 11445 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:49.666162 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:49.661767 12060 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 16: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:49.666774 12067 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:16:49.608752 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:49.671351 11512 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:16:49.671928 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.676398 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:49.680886 16630 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.683156 16631 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:49.682895 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 289496us
I20260709 13:16:49.690676 12066 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:49.690277 11363 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:49.695462 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:49.729948 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.661340 16584 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:49.699576 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 157802us
I20260709 13:16:49.703667 11367 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:49.725358 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:49.731370 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.732008 12067 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 15 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:16:49.705570 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.735700 11515 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:49.705988 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.709357 11511 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:49.646018 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.715863 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:49.737248 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.722304 11369 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:49.738036 11361 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:49.653394 16604 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.730170 11439 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:49.739632 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.695928 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:49.736635 11516 raft_consensus.cc:2368] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:49.744724 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:49.745755 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.746776 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.746976 11515 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:16:49.747149 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:49.748004 11367 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:16:49.752903 12068 raft_consensus.cc:3060] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:49.753420 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.755139 11483 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:49.755925 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:49.750834 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:49.711239 16519 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.756489 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:49.762472 11361 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:49.719031 16649 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.765527 11364 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:16:49.728507 16595 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:49.770754 12068 raft_consensus.cc:2468] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:16:49.730652 16465 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:49.772909 11369 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:16:49.766914 11483 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:49.704013 16567 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.707685 16599 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.765576 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:49.752259 16328 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:49.780005 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:49.752177 16652 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:16:49.749423 16664 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:49.712117 16594 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.753846 16578 raft_consensus.cc:2764] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:16:49.766865 16630 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:49.737381 15852 consensus_queue.cc:1059] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 13:16:49.784017 16460 consensus_meta.cc:287] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.562s	user 0.001s	sys 0.009s
I20260709 13:16:49.775959 16670 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:49.786827 16645 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:49.787070 16519 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:49.785100 15953 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:49.781039 16595 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.782582 16661 raft_consensus.cc:2804] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:49.786007 16652 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.811614 16460 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.830384 11513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48450 (request call id 0) took 2290 ms. Trace:
I20260709 13:16:49.785102 16650 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:49.837235 11513 rpcz_store.cc:276] 0709 13:16:47.540167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.289122 (+748955us) service_pool.cc:224] Handling call
0709 13:16:49.808079 (+1518957us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54001509d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca04735 00007f35bc9fc9fa 00007f35c19c44c9 00007f35baf40c54 00007f35baf40be2 00007f35baf40b11 00007f35baf40a8c 00007f35baf3fd42 00007f35bad3948c 00007f35bad388c6
0709 13:16:49.830291 (+ 22212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702858112}
I20260709 13:16:49.782413 16513 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:16:49.805688 16670 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.828173 16578 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:49.909755 11483 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.510s (SENT) [suppressed 6 similar messages]
I20260709 13:16:49.912671 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:49.919694 12066 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:49.921244 11368 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:49.902797 16519 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.923429 16578 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.931798 12067 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:49.932999 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2601) took 1797 ms. Trace:
I20260709 13:16:49.900091 16652 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:49.933403 11365 rpcz_store.cc:276] 0709 13:16:48.135680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.299098 (+163418us) service_pool.cc:224] Handling call
0709 13:16:48.299327 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.226522 (+927195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.932785 (+706263us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400423954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:49.932948 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2130666368}
I20260709 13:16:49.895867 16656 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:49.780920 16245 consensus_queue.cc:260] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.905447 16661 raft_consensus.cc:697] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:49.908507 16645 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:49.777812 16649 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.909186 16670 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:49.947460 16604 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:49.947707 16467 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:49.950088 12063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1435) took 1626 ms (client timeout 1510 ms). Trace:
I20260709 13:16:49.950438 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:49.955605 12032 leader_election.cc:400] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:49.939420 16515 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:49.979460 16673 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
W20260709 13:16:49.979735 16687 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:16:49.976665 12067 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:49.981783 11442 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:16:49.979221 12032 leader_election.cc:403] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:16:49.957336 12063 rpcz_store.cc:269] 0709 13:16:48.323266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.323773 (+   507us) service_pool.cc:224] Handling call
0709 13:16:49.949913 (+1626140us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54005127d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:49.950009 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1863181696}
W20260709 13:16:49.942240 16680 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:16:49.929275 16519 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:49.797598 16662 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:49.990490 11316 auto_rebalancer.cc:893] Removing replace marker failed: Replica 6e87b213a4554d94825aceec352fc094 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:16:49.990881 11316 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:16:49.991122 11316 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:16:49.982604 16515 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:49.942646 16656 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.989779 16676 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11.
I20260709 13:16:49.948305 16645 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:49.984496 16600 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.958024 16492 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:16:49.977864 16595 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:50.009788 11252 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7a5787304dfe4697baeaeb367ee7dda3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 34a42aa6aa0a4778a532c6433d70b981 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:50.010102 16661 consensus_queue.cc:237] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.011142 16460 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:49.995282 15852 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:50.031281 16686 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
W20260709 13:16:50.067072 16688 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:16:50.067816 16636 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:50.040089 16513 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.068955 16513 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:50.072033 16693 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:16:50.072419 16578 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.076476 16602 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:50.078677 16534 consensus_meta.cc:287] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 13:16:50.003389 16389 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.041703 16594 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:50.112265 16697 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:16:50.230480 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.044756 16467 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:50.333475 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1114) took 1816 ms. Trace:
I20260709 13:16:50.333949 11438 rpcz_store.cc:276] 0709 13:16:48.516796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.603720 (+ 86924us) service_pool.cc:224] Handling call
0709 13:16:48.604005 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.693923 (+1089918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.333219 (+639296us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400399254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:50.333411 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344544000}
I20260709 13:16:50.340037 16513 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.042236 15852 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.316682 16467 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:50.123632 16534 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.382426 16534 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:50.383801 16607 log.cc:927] Time spent T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
W20260709 13:16:50.390378 16568 consensus_meta.cc:287] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.784s	user 0.007s	sys 0.003s
I20260709 13:16:50.396679 16389 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.397565 16649 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:50.395138 16535 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.702s	user 0.006s	sys 0.001s
I20260709 13:16:50.346534 16465 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.346279 16567 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.409822 16465 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.444504 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1975) took 1321 ms. Trace:
I20260709 13:16:50.444978 11518 rpcz_store.cc:276] 0709 13:16:49.123385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.156812 (+ 33427us) service_pool.cc:224] Handling call
0709 13:16:50.444283 (+1287471us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400064fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:50.444432 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183440640}
I20260709 13:16:50.449236 15852 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.451680 16245 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:50.452952 16630 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.002s
I20260709 13:16:50.455778 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37092 (request call id 2043) took 1116 ms. Trace:
I20260709 13:16:50.456212 11368 rpcz_store.cc:276] 0709 13:16:49.339685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.340609 (+   924us) service_pool.cc:224] Handling call
0709 13:16:50.455557 (+1114948us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54002da854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:50.455712 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307412736}
I20260709 13:16:50.458562 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2608) took 1279 ms. Trace:
I20260709 13:16:50.458937 11363 rpcz_store.cc:276] 0709 13:16:49.179012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.195484 (+ 16472us) service_pool.cc:224] Handling call
0709 13:16:49.204639 (+  9155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.729873 (+525234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.458321 (+728448us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b5400165154. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:50.458502 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116389632}
W20260709 13:16:50.464104 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.468400 16460 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:50.468948 16460 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.469363 16483 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:50.469847 16483 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.472280 16653 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:50.472836 16653 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.473417 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1267) took 1111 ms. Trace:
I20260709 13:16:50.473804 12064 rpcz_store.cc:276] 0709 13:16:49.361598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.362341 (+   743us) service_pool.cc:224] Handling call
0709 13:16:50.473264 (+1110923us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b5400741dd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:50.473365 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117889920}
I20260709 13:16:50.477891 16554 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.488945 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:50.492394 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.493952 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.510775 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.513484 12066 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:50.511168 11333 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:50.516387 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.517644 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.519428 11362 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:50.521258 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.521533 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:50.517822 16696 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.524168 16696 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:50.526047 15852 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:50.526037 16696 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:50.517694 16647 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.527685 16696 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:50.527911 16647 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:50.528602 16696 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:50.530383 16696 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:50.531693 16647 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:50.518164 16389 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:50.532852 16389 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:50.438282 16512 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.726s	user 0.003s	sys 0.008s
I20260709 13:16:50.534865 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:50.542302 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:50.542898 11437 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:50.536141 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:50.545145 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.545905 11442 raft_consensus.cc:2410] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:16:50.546698 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.536942 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.548311 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.537134 11439 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:50.538318 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:50.550527 11445 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:50.536607 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.551434 11365 raft_consensus.cc:2368] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:16:50.539027 11364 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:50.538951 11444 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:50.553529 11442 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:16:50.478497 16554 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.421432 16656 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.555805 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.555991 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:50.556996 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:50.559782 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:50.560350 11444 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:50.575235 11368 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:50.576581 11442 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:50.580837 11445 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:50.582645 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:50.533900 15852 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:50.626631 11370 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:50.423425 16467 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:50.529438 16483 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.627508 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:50.640329 11510 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:50.630725 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.631458 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:50.641877 11518 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:16:50.633620 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.565474 16640 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:50.644094 11516 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:50.554261 16479 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:16:50.647506 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.645748 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.660434 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.654639 11444 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:50.662886 11439 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:16:50.663781 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.666977 11444 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:16:50.558010 16512 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:50.574033 16690 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
W20260709 13:16:50.635200 16652 consensus_meta.cc:287] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.689s	user 0.002s	sys 0.010s
I20260709 13:16:50.584234 16377 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:16:50.672214 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:50.675208 16465 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:50.675963 16653 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.700522 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.700838 11436 raft_consensus.cc:1240] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:16:50.681329 16670 consensus_meta.cc:287] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.004s
I20260709 13:16:50.685026 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.685250 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.686110 16479 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:50.686910 16690 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:50.681141 16460 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:50.688441 16679 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.985s	user 0.006s	sys 0.016s
W20260709 13:16:50.689007 16515 consensus_meta.cc:287] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.683s	user 0.005s	sys 0.004s
I20260709 13:16:50.690292 16389 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:50.691656 16654 consensus_meta.cc:287] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.996s	user 0.000s	sys 0.012s
I20260709 13:16:50.692035 15852 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.692658 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.693197 16712 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:50.691579 16652 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.695314 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.696506 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:50.698040 12033 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:50.691489 16704 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:16:50.702924 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.717032 11518 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:50.736913 11484 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:50.729974 11364 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 8.
I20260709 13:16:50.733942 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:50.704247 16479 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:50.734607 16668 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:50.732656 11365 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:50.701511 16660 consensus_meta.cc:287] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.991s	user 0.006s	sys 0.006s
I20260709 13:16:50.742465 12068 raft_consensus.cc:2368] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 8. Current term is 9.
I20260709 13:16:50.742697 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:50.743974 12033 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:50.728097 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.749897 11368 raft_consensus.cc:2368] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:16:50.707872 11437 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:50.705792 16670 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.737121 16278 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:50.753854 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2614) took 1233 ms. Trace:
I20260709 13:16:50.754459 12033 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:50.754328 11366 rpcz_store.cc:276] 0709 13:16:49.520399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.520916 (+   517us) service_pool.cc:224] Handling call
0709 13:16:49.521282 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.521635 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:49.521663 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:49.522162 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:16:49.586412 (+ 64250us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:49.586452 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.586518 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:50.687903 (+1101385us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:50.745200 (+ 57297us) raft_consensus.cc:1685] finished
0709 13:16:50.745221 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.753782 (+  8561us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22922,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":14135,"spinlock_wait_cycles":3376896,"thread_start_us":63952,"threads_started":1,"wal-append.queue_time_us":130701}
I20260709 13:16:50.756798 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1443) took 1163 ms. Trace:
I20260709 13:16:50.757170 12062 rpcz_store.cc:276] 0709 13:16:49.593797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.593939 (+   142us) service_pool.cc:224] Handling call
0709 13:16:49.594617 (+   678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.594976 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:49.595003 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:49.932977 (+337974us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b2c000a6788. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:49.933291 (+   314us) log.cc:844] Serialized 26 byte log entry
0709 13:16:50.357253 (+423962us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:50.357289 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.357348 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:50.751734 (+394386us) raft_consensus.cc:1685] finished
0709 13:16:50.751753 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.756736 (+  4983us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":35892096,"thread_start_us":423694,"threads_started":1,"wal-append.queue_time_us":604582}
I20260709 13:16:50.758189 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:50.737740 16690 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:50.760054 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.731976 16515 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:50.793330 16649 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.798058 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.798923 11364 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:50.767175 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:50.762495 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:50.818230 11510 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:50.773415 11365 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:16:50.820017 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1269) took 1457 ms. Trace:
I20260709 13:16:50.794005 16649 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.820485 12060 rpcz_store.cc:276] 0709 13:16:49.362160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.362276 (+   116us) service_pool.cc:224] Handling call
0709 13:16:50.819793 (+1457517us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54004a6fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:50.819945 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672819072}
I20260709 13:16:50.821274 16599 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:50.822310 16599 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.822882 16649 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.822783 16599 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.824285 16648 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.824883 16599 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.827967 16592 consensus_queue.cc:260] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.828886 16535 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:50.829239 16592 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:50.829689 16535 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:50.830219 16535 consensus_queue.cc:1243] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:16:50.831262 16535 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:50.831972 16535 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:50.832947 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35176 (request call id 950) took 1086 ms. Trace:
I20260709 13:16:50.832716 16535 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:50.773603 12033 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:50.745042 16712 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:16:50.839854 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 66360.2us
I20260709 13:16:50.776836 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:50.817394 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1434) took 2495 ms. Trace:
I20260709 13:16:50.828521 11409 leader_election.cc:400] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:16:50.841501 11409 leader_election.cc:403] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:16:50.753509 16630 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:50.740365 16036 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:50.841162 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:50.836323 11367 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:50.753580 16530 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.753639 16414 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:50.833330 12065 rpcz_store.cc:276] 0709 13:16:49.750525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.757699 (+  7174us) service_pool.cc:224] Handling call
0709 13:16:49.757989 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.785570 (+ 27581us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:49.808299 (+ 22729us) log.cc:844] Serialized 14 byte log entry
0709 13:16:50.467763 (+659464us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b1800065500. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35bc5a64bc 00007f35b98f448b 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e23a
0709 13:16:50.758976 (+291213us) log.cc:844] Serialized 14 byte log entry
0709 13:16:50.764780 (+  5804us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:50.764833 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:50.764882 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.832720 (+ 67838us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b5400654fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:50.832768 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.832891 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5488,"prepare.queue_time_us":22588,"prepare.run_cpu_time_us":9487,"prepare.run_wall_time_us":60444,"raft.queue_time_us":8761,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"spinlock_wait_cycles":93987456,"thread_start_us":383783,"threads_started":2,"wal-append.queue_time_us":138199}
I20260709 13:16:50.843936 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:50.759716 16502 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:50.846416 11519 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:50.753333 16676 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:50.736130 16654 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.762142 16668 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:50.841444 12061 rpcz_store.cc:276] 0709 13:16:48.322476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.322607 (+   131us) service_pool.cc:224] Handling call
0709 13:16:48.322883 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.115606 (+792723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.817151 (+1701545us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54004e7854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:50.817329 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793782144}
I20260709 13:16:50.855806 16535 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:50.856756 11443 raft_consensus.cc:2368] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10.
W20260709 13:16:50.857981 11443 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41514 (request call id 1120) took 1543 ms (client timeout 1609 ms). Trace:
W20260709 13:16:50.858371 11443 rpcz_store.cc:269] 0709 13:16:49.314252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.315621 (+  1369us) service_pool.cc:224] Handling call
0709 13:16:50.857809 (+1542188us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54002094d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:50.857928 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3470990208}
I20260709 13:16:50.839781 16479 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:50.847612 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 71132.1us
I20260709 13:16:50.847898 11409 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:50.863970 16711 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:16:50.864499 16534 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:50.864420 16592 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:50.865011 16534 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:50.865391 16534 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:50.849150 11514 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:50.848482 16712 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:50.836651 16600 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:16:50.841579 16155 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:16:50.873304 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 245956us
I20260709 13:16:50.851702 16676 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.852445 16530 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.854225 16554 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.883782 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.854578 16654 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.741777 16709 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
I20260709 13:16:50.851745 16410 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:50.890659 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.891518 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.891755 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.892155 11436 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:50.868924 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.753404 16650 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:16:50.753404 16412 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:16:50.894800 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:50.871101 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:50.871575 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.876214 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:50.896298 11440 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:50.876355 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.878219 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:50.897835 12062 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:50.879258 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.860273 16479 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:50.741348 16660 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.881287 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.778865 16665 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:16:50.850340 16250 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:50.885252 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.886065 11364 pending_rounds.cc:85] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 6
I20260709 13:16:50.914356 11364 pending_rounds.cc:107] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:16:50.914667 11364 raft_consensus.cc:2889] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:50.915542 16652 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:50.886541 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:50.886912 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.917238 11444 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:50.888789 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:50.918130 11439 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:50.872390 16521 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.918947 16521 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.921204 16521 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:50.880472 11437 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:50.869395 16649 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:16:50.878101 16502 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:50.923741 16502 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.924868 16660 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.873194 16578 consensus_queue.cc:1059] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:16:50.906180 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.926530 16578 raft_consensus.cc:3055] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:50.926901 16578 raft_consensus.cc:740] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:50.927062 11513 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:50.927938 11366 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:50.898492 16554 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.933256 16554 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.919545 16510 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:16:50.936218 16670 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:50.937274 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:50.938326 16512 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:50.884021 16690 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.942970 16690 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:50.950698 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:50.951613 11370 raft_consensus.cc:2410] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:16:50.882479 16600 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:50.901477 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:50.909731 16530 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:50.961884 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 60482.9us
I20260709 13:16:50.910631 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:50.959739 11366 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:50.935302 16554 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:50.888427 16656 consensus_queue.cc:1059] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:16:50.899598 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:50.905580 16410 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:50.965662 16483 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:50.980063 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 80541.2us
I20260709 13:16:50.959898 12065 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:50.965863 11363 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:50.978468 11438 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 8.
I20260709 13:16:50.906034 16696 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.966636 16654 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:50.966881 16649 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:51.005525 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.012409 11515 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:50.953176 16512 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.017809 16512 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.019917 16512 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.022349 16534 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.023500 11443 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:51.024842 16599 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.025341 16599 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.028285 16600 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:50.993639 16483 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.031497 16483 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:51.033360 16676 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.035414 16696 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.002761 16578 consensus_queue.cc:260] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.001946 16712 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.038165 16515 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:51.002591 16594 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:51.040870 11370 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:51.013878 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.015127 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.051667 12065 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:51.053398 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.057090 12064 raft_consensus.cc:1240] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:16:51.057216 12065 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:16:51.059969 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 89872us
I20260709 13:16:51.066299 12067 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:16:51.036875 16245 consensus_meta.cc:287] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.585s	user 0.004s	sys 0.005s
I20260709 13:16:51.068269 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.069720 11511 raft_consensus.cc:2410] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:16:51.071913 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:51.072513 11515 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:51.027259 16599 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:51.086651 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.070879 16710 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 13:16:51.091599 12067 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:51.001538 16535 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:51.107048 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.108201 11515 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:51.109190 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:51.115468 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 53295.2us
I20260709 13:16:51.090138 11363 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 10.
I20260709 13:16:51.128851 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:51.133890 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.137118 11517 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:51.138549 12065 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:51.125509 11335 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.143265 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.143898 12065 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:51.148352 16592 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:51.148854 16592 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.150874 16592 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.162180 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.164180 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.159675 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:51.140189 16535 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.166847 16535 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.133360 16684 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:51.167964 16684 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.170224 16684 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.170890 16515 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.171276 16708 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:16:51.171448 16515 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.174012 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.055639 16649 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.055431 16654 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:51.175374 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49945.6us
I20260709 13:16:51.164796 11517 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:51.092259 16245 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:51.173529 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:51.017473 16410 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.173514 16515 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:51.180583 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.183562 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.155965 16650 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.140923 16722 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:51.140771 16328 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.176296 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:51.188297 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 188807us
I20260709 13:16:51.182698 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:51.140492 16721 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:51.210069 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.211057 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.180193 16554 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:51.210896 16535 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:51.183137 16592 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:51.242781 16592 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.245003 16592 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.215584 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.203569 16250 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:51.245988 16250 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:51.246912 16250 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.219039 16515 raft_consensus.cc:2764] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:16:51.218868 16328 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.251907 16328 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.252686 16534 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.185006 16709 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:51.186547 16245 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.207294 16640 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:51.226754 16654 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.228299 16649 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.231314 16655 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:51.210089 16721 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:51.239145 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.240878 16483 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.249540 16412 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:51.221320 16716 log.cc:927] Time spent T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:16:51.259025 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.217830 16660 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.212437 16650 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.274463 16660 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.276714 16594 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.277222 16594 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:51.278014 16660 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:51.279024 12065 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:51.280833 16721 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.282951 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.283721 16592 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:51.284135 16650 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.260380 16515 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:16:51.284811 16650 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:51.265405 16278 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:16:51.284468 16709 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:16:51.285565 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:51.289422 16709 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:51.289812 16709 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.290010 16483 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.279106 16245 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.290252 16592 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:51.292968 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.293784 12065 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:51.297577 16578 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:51.307719 16245 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.308316 16245 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.310483 16245 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:51.285517 16650 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
W20260709 13:16:51.312346 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.313378 16709 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:51.323303 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.324162 12065 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:51.324960 15430 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.324417 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.330400 12066 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:51.325433 15430 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.340583 15430 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.338856 16414 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:51.346014 16336 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:51.196370 11365 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:16:51.189991 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.462165 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.465528 12065 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:16:51.432049 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:51.496039 11436 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:51.352713 16414 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.461581 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.498930 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:51.460366 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:51.500939 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.502383 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.408429 11362 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:51.402853 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:51.459508 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.503325 11365 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:51.461647 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.412755 11366 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:16:51.502236 16650 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:51.449359 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.413753 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.431133 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.440213 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:51.449746 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:51.530404 11440 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:51.401026 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.523262 16650 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:51.507831 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 47864.3us
I20260709 13:16:51.353955 16336 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:51.430048 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 218925us
I20260709 13:16:51.543924 11511 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:51.539609 16650 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:51.544765 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.545075 16742 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:51.548611 11369 raft_consensus.cc:1813] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Handling vote request from an unknown peer 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:51.549350 11369 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:51.553272 16534 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:51.560055 11365 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:51.561527 11365 raft_consensus.cc:1240] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:51.563980 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2795) took 1023 ms. Trace:
I20260709 13:16:51.357223 16594 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.563337 16336 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.564496 11441 rpcz_store.cc:276] 0709 13:16:50.540831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.540927 (+    96us) service_pool.cc:224] Handling call
0709 13:16:50.541232 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:50.541573 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:50.970372 (+428799us) log.cc:844] Serialized 14 byte log entry
0709 13:16:51.429127 (+458755us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:51.430404 (+  1277us) log.cc:844] Serialized 52 byte log entry
0709 13:16:51.430635 (+   231us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:51.431080 (+   445us) log.cc:844] Serialized 14 byte log entry
0709 13:16:51.431258 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.431368 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:51.563460 (+132092us) raft_consensus.cc:1685] finished
0709 13:16:51.563484 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:51.563750 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24164,"prepare.queue_time_us":187032,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":192240,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":41723,"spinlock_wait_cycles":7264640,"thread_start_us":760034,"threads_started":2,"wal-append.queue_time_us":278896}
I20260709 13:16:51.573232 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.574473 11364 raft_consensus.cc:2410] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:16:51.575529 11444 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:16:51.575805 12033 leader_election.cc:400] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:16:51.576303 12033 leader_election.cc:403] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:16:51.367996 16709 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.576543 11366 raft_consensus.cc:1275] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:16:51.578471 11363 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:51.551184 16414 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.579708 11438 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:51.579861 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:51.581399 16630 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:16:51.582010 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 181517us
I20260709 13:16:51.584055 15852 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:51.584566 15852 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:51.585021 15852 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:51.587702 16377 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:51.588460 16377 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.579190 16709 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.567312 11368 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:51.613554 11365 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:51.615142 16594 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.618083 16336 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:51.620478 11516 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:51.627871 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:51.628760 12065 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:16:51.593948 16592 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:16:51.635493 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.635545 11369 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:16:51.635895 16742 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.645874 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.646945 16654 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:51.647814 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:51.648795 16654 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:51.649408 16654 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:51.650921 16377 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.650823 16654 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.653532 11335 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:51.620473 16521 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:51.592908 16590 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:16:51.656811 16708 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:16:51.657562 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.658059 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:51.658412 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:51.658996 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:51.659134 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.673096 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.673606 11515 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:51.676128 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 100872us
I20260709 13:16:51.677218 16709 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:51.677841 16709 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:51.678324 16709 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:51.680392 16569 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:51.686091 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:51.669442 16521 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:51.690106 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:51.677320 16707 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:51.690745 16521 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.695760 11516 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:51.696874 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.687255 16377 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:51.700318 11514 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36658 (request call id 1980) took 1693 ms. Trace:
I20260709 13:16:51.700742 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.705451 16650 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.707578 16534 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.713236 16535 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.723661 16467 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:51.724156 16521 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:51.726125 16530 raft_consensus.cc:3055] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:51.726583 16530 raft_consensus.cc:740] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:51.720091 16377 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:51.727483 16377 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:51.715693 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.698251 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1277) took 1209 ms. Trace:
I20260709 13:16:51.719129 11444 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:51.720155 11514 rpcz_store.cc:276] 0709 13:16:50.006583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.006739 (+   156us) service_pool.cc:224] Handling call
0709 13:16:50.006993 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.873706 (+866713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.699753 (+826047us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b540031f954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:51.700111 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931897600}
W20260709 13:16:51.736771 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 403416us
I20260709 13:16:51.730005 12068 rpcz_store.cc:276] 0709 13:16:50.488441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.488730 (+   289us) service_pool.cc:224] Handling call
0709 13:16:51.697793 (+1209063us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400751054. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:51.698024 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581248256}
I20260709 13:16:51.741677 11511 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:51.747692 11519 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:51.753036 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:51.755756 16530 consensus_queue.cc:260] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.757021 16530 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:51.769135 16733 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:51.770042 16733 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:51.772660 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.773288 11444 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:51.774366 11442 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:16:51.776121 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.776324 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:51.779038 11335 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:51.779708 16653 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:51.780596 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 80044.8us
I20260709 13:16:51.781263 16630 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:51.780831 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.783874 16602 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:51.784269 16373 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:51.744241 16708 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:51.816493 11436 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:16:51.809765 11515 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:51.812129 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 94476.3us
I20260709 13:16:51.812819 12065 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:51.771185 16733 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:51.825441 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46536.5us
I20260709 13:16:51.723664 16535 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.799839 16653 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:51.835634 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 120183us
I20260709 13:16:51.842243 16653 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:51.872841 16377 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.875000 15698 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:16:51.875780 16578 consensus_meta.cc:287] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.001s
I20260709 13:16:51.842923 16534 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.879065 16534 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.879493 16532 raft_consensus.cc:2764] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:16:51.889140 16535 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:51.889751 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52763.1us
I20260709 13:16:51.903560 16336 consensus_queue.cc:1048] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:51.876888 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:51.910533 16742 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:51.910964 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:51.836051 15852 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.913690 11510 raft_consensus.cc:2393] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:16:51.914814 16753 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:16:51.925295 15852 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.926755 11516 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:51.930518 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:51.932111 11408 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:51.934511 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:51.936760 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:51.937709 16532 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:51.939734 16377 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.941083 16535 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:51.941872 16535 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:51.942538 16535 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:51.943061 16535 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:51.916118 16709 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:51.944486 16535 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.946599 16709 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.948134 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.951538 16742 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:51.952047 16742 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.952431 16742 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:51.951767 16650 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:51.928182 16707 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:51.957224 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.958627 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.960201 11439 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:51.961452 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:51.962044 11437 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:51.964371 11445 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:16:51.965586 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 130501us
I20260709 13:16:51.970198 15698 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:51.971127 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.971796 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.972687 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.972977 16521 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.973438 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:51.974591 16656 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:51.974947 16751 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.247s	user 0.002s	sys 0.000s
I20260709 13:16:51.975083 16656 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:51.976313 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:51.978372 11408 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:51.978935 11268 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39460 (request call id 366) took 23086 ms (client timeout 15000 ms). Trace:
I20260709 13:16:51.979779 16732 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:16:51.981732 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 58486.7us
I20260709 13:16:51.956423 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.928347 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:51.986898 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47618.9us
I20260709 13:16:51.973843 12034 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:51.991297 12061 raft_consensus.cc:2393] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:16:51.993180 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 215719us
I20260709 13:16:51.990577 16732 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:52.004029 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:52.002039 11333 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:52.007252 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:51.973508 16521 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.008108 15852 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:16:52.017455 11438 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:52.015698 16092 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:51.996358 16532 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:16:52.028023 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:52.028003 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:52.008004 16534 consensus_queue.cc:1059] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:16:51.971189 15698 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.027565 16510 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:16:52.051692 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 50933us
I20260709 13:16:52.049681 11440 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:52.059218 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:52.057466 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:52.057092 16594 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:16:52.064051 16594 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:52.064401 16594 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:51.979389 11268 rpcz_store.cc:269] 0709 13:16:28.892320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.892531 (+   211us) service_pool.cc:224] Handling call
0709 13:16:51.978681 (+23086150us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:48.637430 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:48.637946 (+   516us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:48.637981 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:16:48.637998 (+    17us) tablet.cc:662] Decoding operations
0709 13:16:48.638368 (+   370us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:48.638442 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:48.638466 (+    24us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:16:48.638767 (+   301us) tablet.cc:701] Row locks acquired
0709 13:16:48.638784 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:16:48.638934 (+   150us) write_op.cc:270] Start()
0709 13:16:48.639049 (+   115us) write_op.cc:276] Timestamp: P: 1783603008638890 usec, L: 0
0709 13:16:48.639069 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:16:48.644675 (+  5606us) log.cc:844] Serialized 917 byte log entry
0709 13:16:49.830760 (+1186085us) op_driver.cc:464] REPLICATION: finished
0709 13:16:50.419787 (+589027us) write_op.cc:301] APPLY: starting
0709 13:16:50.421642 (+  1855us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:50.456991 (+ 35349us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:50.457099 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:50.458297 (+  1198us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:50.458493 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:16:50.458537 (+    44us) write_op.cc:312] APPLY: finished
0709 13:16:50.458737 (+   200us) log.cc:844] Serialized 33 byte log entry
0709 13:16:50.458909 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:50.459008 (+    99us) write_op.cc:454] Released schema lock
0709 13:16:50.459055 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:50.537996 (+ 78941us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:50.539188 (+  1192us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1988520,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":580461,"apply.run_cpu_time_us":4463,"apply.run_wall_time_us":95494,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12479,"num_ops":2,"prepare.queue_time_us":362505,"prepare.run_cpu_time_us":17569,"prepare.run_wall_time_us":453621,"raft.queue_time_us":259052,"raft.run_cpu_time_us":15928,"raft.run_wall_time_us":261384,"replication_time_us":1155458,"thread_start_us":1097213,"threads_started":4,"wal-append.queue_time_us":527830,"wal-append.run_cpu_time_us":18327,"wal-append.run_wall_time_us":2318533}]]}
I20260709 13:16:52.065248 16594 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:52.068168 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.064988 16676 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:16:52.079730 11362 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:52.080722 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:52.081486 12069 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:52.090099 16656 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:52.090981 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:52.091444 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:52.092593 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 118893us
I20260709 13:16:52.093636 16650 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:52.094321 16650 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:52.096743 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:52.072559 11333 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:52.104208 11510 raft_consensus.cc:2393] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:52.095359 16650 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:52.106051 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:52.106575 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.107486 12032 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:52.108348 16092 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:52.110055 16092 raft_consensus.cc:2764] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:16:52.113895 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:52.114907 16758 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:52.117190 16590 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:52.118090 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:52.119975 12068 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:52.121331 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:52.123888 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:52.119863 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:52.132009 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 69384us
I20260709 13:16:52.086927 16653 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.149029 16653 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:52.102746 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:52.152680 16521 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:52.066522 16594 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:52.155982 11438 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
W20260709 13:16:52.160449 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 63912.7us
I20260709 13:16:52.161365 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:52.162490 16656 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:52.135561 11437 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:52.115213 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:52.165267 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 56129.7us
I20260709 13:16:52.167704 11366 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:52.052517 16630 consensus_queue.cc:1059] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:16:52.143637 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:52.096573 16712 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:16:52.145259 12060 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:16:52.170311 11440 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:52.138921 12069 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:52.184046 12065 raft_consensus.cc:1240] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
W20260709 13:16:52.188613 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 73464.8us
I20260709 13:16:52.176915 16377 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:52.240175 11363 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:52.185802 16653 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:52.245947 16653 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:52.247989 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 113666us
I20260709 13:16:52.188493 16534 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:52.248970 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:52.249361 16510 raft_consensus.cc:3055] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:52.249701 16532 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:52.250509 16377 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:52.251526 16377 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:52.242097 16535 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:52.254328 16535 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.256371 16535 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:52.244958 16602 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:52.245256 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 52633.8us
I20260709 13:16:52.258455 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:52.260694 16665 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:52.260970 16092 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:52.243559 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:52.264493 16707 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:52.264846 16602 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.267577 16602 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:52.268656 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:52.269702 16656 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:52.272106 16656 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:52.273276 16656 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:52.274598 16656 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.276093 16655 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:52.277004 16630 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:52.276805 16655 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.278354 16707 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:52.279567 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 114648us
I20260709 13:16:52.280388 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:52.249776 16510 raft_consensus.cc:740] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:52.283586 16578 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:52.284415 16578 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:52.286144 16578 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:52.248157 16653 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:52.303756 16707 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:52.283183 16510 consensus_queue.cc:260] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:52.321235 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:52.322952 12069 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:16:52.325934 16665 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:52.306157 16602 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:52.279263 16655 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
W20260709 13:16:52.717315 16762 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.211s	user 0.002s	sys 0.002s
W20260709 13:16:52.719879 16764 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.187s	user 0.002s	sys 0.001s
I20260709 13:16:52.862893 16650 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.863785 16650 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:52.866232 16650 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:52.866784 16650 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:53.107040 16763 log.cc:927] Time spent T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 13:16:53.315743 16707 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:52.867360 16650 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:53.394444 16650 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:53.395082 16650 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:53.316699 16707 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:53.745450 16707 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:53.746172 16707 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.746852 16707 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:53.749790 16707 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:53.672397 11299 debug-util.cc:398] Leaking SignalData structure 0x7b0800275b80 after lost signal to thread 10260
I20260709 13:16:53.696985 15852 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:53.869150 15852 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:53.866948 11408 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:53.871122 15852 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:53.796589 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1291) took 1612 ms. Trace:
I20260709 13:16:53.871718 12060 rpcz_store.cc:276] 0709 13:16:52.184180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.191431 (+  7251us) service_pool.cc:224] Handling call
0709 13:16:52.191761 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:53.796101 (+1604340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:53.796145 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:53.796203 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:53.796239 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.796318 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.796476 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:53.743175 16650 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:53.856559 16467 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.764s	user 0.006s	sys 0.038s
W20260709 13:16:53.873337 16467 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.781s	user 0.007s	sys 0.038s
I20260709 13:16:53.872538 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:53.704780 16653 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.876271 16653 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:53.868263 16707 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.640545 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:53.878926 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 1983) took 1550 ms. Trace:
I20260709 13:16:53.880015 11510 rpcz_store.cc:276] 0709 13:16:52.328793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.328913 (+   120us) service_pool.cc:224] Handling call
0709 13:16:53.878842 (+1549929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:53.878662 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2651) took 1131 ms. Trace:
I20260709 13:16:53.882472 16653 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:53.902346 16653 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:53.881886 11365 rpcz_store.cc:276] 0709 13:16:52.746855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.872244 (+1125389us) service_pool.cc:224] Handling call
0709 13:16:53.878431 (+  6187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:53.906163 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:53.906761 11365 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:53.907202 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2649) took 1160 ms. Trace:
I20260709 13:16:53.907488 11365 rpcz_store.cc:276] 0709 13:16:52.746398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.905879 (+1159481us) service_pool.cc:224] Handling call
0709 13:16:53.907151 (+  1272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:53.908345 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:53.908999 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 1299) took 1664 ms. Trace:
I20260709 13:16:53.909281 11365 rpcz_store.cc:276] 0709 13:16:52.244957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.908101 (+1663144us) service_pool.cc:224] Handling call
0709 13:16:53.908925 (+   824us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:53.885540 16650 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:53.879911 16707 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:53.888693 11362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.772s	user 0.017s	sys 0.028s
W20260709 13:16:53.924151 11362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.807s	user 0.018s	sys 0.029s
W20260709 13:16:53.902943 11444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.877s	user 0.048s	sys 0.073s
W20260709 13:16:53.924616 11444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.901s	user 0.049s	sys 0.073s
W20260709 13:16:53.883424 11334 connection.cc:256] RPC call timeout handler was delayed by -1.5415310210000825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 13:16:53.925558 11334 connection.cc:256] RPC call timeout handler was delayed by -1.4342793790001451s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 13:16:53.925199 16707 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:53.929133 11369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.745s	user 0.009s	sys 0.029s
W20260709 13:16:53.929498 11369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.746s	user 0.009s	sys 0.029s
I20260709 13:16:53.930145 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 2064) took 1905 ms. Trace:
I20260709 13:16:53.930480 11369 rpcz_store.cc:276] 0709 13:16:52.024566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.024677 (+   111us) service_pool.cc:224] Handling call
0709 13:16:52.047861 (+ 23184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.054471 (+  6610us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:52.134627 (+ 80156us) log.cc:844] Serialized 14 byte log entry
0709 13:16:53.929786 (+1795159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:53.929838 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:53.929880 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.929941 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.930062 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43520,"prepare.queue_time_us":52557,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":52730,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3286,"thread_start_us":1745851,"threads_started":1}
W20260709 13:16:53.934512 11333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.766s	user 0.029s	sys 0.027s
W20260709 13:16:53.935071 11333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.767s	user 0.030s	sys 0.028s
I20260709 13:16:53.935829 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1294) took 1753 ms. Trace:
I20260709 13:16:53.936866 11369 rpcz_store.cc:276] 0709 13:16:52.181929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.932137 (+1750208us) service_pool.cc:224] Handling call
0709 13:16:53.932508 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:53.932685 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:53.932708 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:53.932746 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:53.932776 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.932824 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.935732 (+  2908us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:53.971858 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:53.972616 12066 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
W20260709 13:16:53.974895 11441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.885s	user 0.025s	sys 0.026s
W20260709 13:16:53.975281 11441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.886s	user 0.025s	sys 0.026s
I20260709 13:16:53.981410 16676 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.982021 16676 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:53.984107 16676 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:53.984910 16676 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:53.985461 16676 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:53.987425 16676 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:53.988449 15698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.842s	user 0.009s	sys 0.009s
W20260709 13:16:53.988823 15698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.842s	user 0.009s	sys 0.009s
I20260709 13:16:54.010311 16467 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:54.010965 16467 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.013198 16467 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.014708 16707 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.015230 16707 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:53.938457 16653 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.017175 16707 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:54.024642 16786 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.027375 16707 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.027966 16707 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.003260 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1290) took 1883 ms. Trace:
I20260709 13:16:54.031457 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.032605 16467 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.033200 16467 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.025435 16786 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.035529 16467 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:54.030427 12065 rpcz_store.cc:276] 0709 13:16:52.133465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.133595 (+   130us) service_pool.cc:224] Handling call
0709 13:16:52.183875 (+ 50280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.184448 (+   573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.003010 (+1818562us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b54004e6e50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:54.003192 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100848384}
I20260709 13:16:54.037695 16786 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.038391 16707 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:54.038792 11519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.896s	user 0.031s	sys 0.039s
W20260709 13:16:54.039148 11519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.897s	user 0.031s	sys 0.039s
I20260709 13:16:54.039664 15852 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.067956 15852 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.038936 16707 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:54.070235 16707 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.070621 16707 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:54.073594 16786 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:16:54.089591 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.092100 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.092751 11510 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 19.
I20260709 13:16:54.086870 16653 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.096863 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 2065) took 1994 ms. Trace:
I20260709 13:16:54.096721 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.098634 11514 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:54.095306 16785 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.092643 16786 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.107879 16782 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.108457 16782 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.109319 16786 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.110522 16782 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.111500 16796 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.112025 16796 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.114176 16796 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.114671 16796 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.118266 16797 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.115172 16796 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.118857 16797 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.120765 16796 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.121353 16796 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.121814 16797 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:54.122753 11409 connection.cc:256] RPC call timeout handler was delayed by -1.0734972089999246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 13:16:54.123127 16797 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:54.125735 16755 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.121874 16796 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.126278 16755 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.127658 16759 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.132332 16803 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:54.134634 16797 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.136687 16797 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.137223 16797 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.137688 16797 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.139206 16796 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.139784 16796 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.129668 16785 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.140600 16787 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:54.136777 12067 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:54.143532 11365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1293) took 1948 ms. Trace:
I20260709 13:16:54.149425 16803 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.153611 16803 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.143955 11365 rpcz_store.cc:276] 0709 13:16:52.181679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.909836 (+1728157us) service_pool.cc:224] Handling call
0709 13:16:53.910132 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:53.910375 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:53.915619 (+  5244us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.129671 (+214052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:54.129726 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:54.129770 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.129839 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.129966 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":517,"prepare.queue_time_us":128409,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":160511,"spinlock_wait_cycles":43874176,"thread_start_us":191906,"threads_started":3,"wal-append.queue_time_us":120887}
I20260709 13:16:54.160722 11333 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.148480 16759 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.141680 16785 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.164062 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:54.164381 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 297569us
I20260709 13:16:54.179733 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.150110 16092 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.195277 11510 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
W20260709 13:16:54.198827 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44043.3us
I20260709 13:16:54.182225 16797 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.181258 16755 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.182456 16786 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.201835 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.163547 16512 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.197296 16803 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.217634 16803 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:54.218351 11361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.135s	user 0.008s	sys 0.015s
W20260709 13:16:54.218677 11361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.135s	user 0.008s	sys 0.015s
I20260709 13:16:54.203913 16805 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.202067 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.221123 16789 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.221647 16789 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.226531 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.226670 16512 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.227427 12069 raft_consensus.cc:2410] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:16:54.197407 16092 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:54.227859 12069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 967) took 1900 ms (client timeout 2283 ms). Trace:
W20260709 13:16:54.228214 12069 rpcz_store.cc:269] 0709 13:16:52.409473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.226244 (+1816771us) service_pool.cc:224] Handling call
0709 13:16:54.227804 (+  1560us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:54.229979 16793 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.230618 16793 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.204396 16787 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:54.232111 16787 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.232470 16787 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:54.221807 16803 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.210892 16800 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.216096 16786 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:54.206879 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 133685us
I20260709 13:16:54.220525 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1295) took 2030 ms. Trace:
I20260709 13:16:54.235145 12067 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:54.235450 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.150019 16796 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.232409 16785 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:54.247468 16785 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.247879 16785 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:54.250533 16761 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.227s	user 0.000s	sys 0.003s
I20260709 13:16:54.239501 16800 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.243544 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1293) took 1912 ms. Trace:
I20260709 13:16:54.243064 11369 rpcz_store.cc:276] 0709 13:16:52.190001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.937856 (+1747855us) service_pool.cc:224] Handling call
0709 13:16:53.942665 (+  4809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:53.943046 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:54.077057 (+134011us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.220168 (+143111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:54.220217 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:54.220257 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.220359 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.220470 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":95870,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":12816,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":249939,"threads_started":2,"wal-append.queue_time_us":161870}
I20260709 13:16:54.260183 11510 raft_consensus.cc:2410] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:16:54.258805 12067 rpcz_store.cc:276] 0709 13:16:52.446786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.136354 (+1689568us) service_pool.cc:224] Handling call
0709 13:16:54.136658 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:54.243245 (+106587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.243484 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:54.263554 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 88767.6us
I20260709 13:16:54.263306 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:54.268286 11510 raft_consensus.cc:2368] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:54.263325 16092 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.269258 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.265045 16803 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.275729 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.272908 16092 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.279817 16092 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.281800 16787 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.283128 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.283893 12060 raft_consensus.cc:2393] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:54.284240 11365 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
I20260709 13:16:54.284565 16803 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.285358 16512 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.285943 16759 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.234771 16805 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.287966 16805 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:52.327224 16665 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.292860 16707 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.295737 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.296504 11510 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:54.304157 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2588) took 2212 ms. Trace:
I20260709 13:16:54.263388 16797 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:16:54.304549 11516 rpcz_store.cc:276] 0709 13:16:52.085053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.085201 (+   148us) service_pool.cc:224] Handling call
0709 13:16:52.085530 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.085806 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:52.447144 (+361338us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.297026 (+1849882us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:54.297093 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:54.297138 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.297203 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.297312 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1793571,"prepare.queue_time_us":1940086,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":57,"raft.queue_time_us":24514,"thread_start_us":392357,"threads_started":2}
I20260709 13:16:54.307385 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.313524 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.314636 15852 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.317322 12060 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:16:54.281869 16092 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.317692 12060 pending_rounds.cc:85] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 8
I20260709 13:16:54.320760 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.321913 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.322619 11516 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:54.312750 16774 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.320089 12060 pending_rounds.cc:107] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:16:54.333313 11510 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:54.336443 16774 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.322343 16092 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.337900 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.316011 16467 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:54.378335 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1292) took 2196 ms. Trace:
I20260709 13:16:54.385993 15852 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.338655 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:54.387349 11369 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:54.384654 16796 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.373383 16811 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.386350 11366 rpcz_store.cc:276] 0709 13:16:52.181457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.232229 (+ 50772us) service_pool.cc:224] Handling call
0709 13:16:52.242697 (+ 10468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.257550 (+2014853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.257589 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.258048 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.309991 (+ 51943us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:54.310031 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.310109 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.378079 (+ 67970us) raft_consensus.cc:1685] finished
0709 13:16:54.378110 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.378254 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":51594,"threads_started":1,"wal-append.queue_time_us":113972}
I20260709 13:16:54.388780 16811 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.389530 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.389912 16650 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.390545 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.378893 16808 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.391307 16808 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.393303 16808 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.393033 15852 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.377259 16656 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.394991 16656 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.395934 15852 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.396389 15852 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.401118 15852 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:54.330848 16709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.235s	user 0.017s	sys 0.032s
I20260709 13:16:54.321054 16707 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:54.336246 12060 raft_consensus.cc:2889] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:54.404111 11516 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:54.393481 16712 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.405989 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:54.412364 11484 connection.cc:256] RPC call timeout handler was delayed by -1.155589444999805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
I20260709 13:16:54.373837 16812 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:54.405689 16709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.308s	user 0.020s	sys 0.042s
I20260709 13:16:54.390837 16811 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.327523 16785 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.390484 16650 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.421343 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.456549 11516 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:54.457525 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:54.421618 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.458797 11516 raft_consensus.cc:2368] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11.
I20260709 13:16:54.444279 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:54.405432 16467 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.437341 16665 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.462321 16665 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.464438 16665 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.412534 16092 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.420485 16707 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.461390 11366 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
W20260709 13:16:54.457800 16708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.825s	user 0.013s	sys 0.022s
I20260709 13:16:54.461275 16467 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:54.442126 16811 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.475266 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.475880 12066 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.477099 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.475772 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.444825 16656 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.478854 12061 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:54.479429 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.480132 12065 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:54.481801 11366 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:54.494071 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.495764 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.506446 11516 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:54.474237 16707 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.498100 16602 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.508211 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.509039 11516 raft_consensus.cc:2368] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:54.509166 16602 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.509924 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.420521 16812 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.420087 16712 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.510876 11516 raft_consensus.cc:2410] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:16:54.511500 11366 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:54.512625 12066 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:54.526230 16785 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.480329 16811 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.560213 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.565147 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:54.595609 11436 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.313s	user 0.016s	sys 0.019s
I20260709 13:16:54.591831 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.564931 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.609851 11369 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
W20260709 13:16:54.610239 11436 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.328s	user 0.017s	sys 0.019s
I20260709 13:16:54.611239 11513 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:54.613082 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.613687 11369 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:54.594336 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.586159 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.527477 16707 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.618314 11366 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
W20260709 13:16:54.531538 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 74283.7us
W20260709 13:16:54.596204 16661 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.466s	user 0.012s	sys 0.031s
I20260709 13:16:54.580549 16812 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.530342 16092 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.625185 16811 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.625664 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.626284 11364 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:54.627074 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:54.628373 11367 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.553s	user 0.040s	sys 0.043s
I20260709 13:16:54.564529 16467 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.629292 11510 raft_consensus.cc:2368] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 19. Current term is 20.
W20260709 13:16:54.616369 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 88289.1us
I20260709 13:16:54.621636 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.629971 16510 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.631742 11516 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:16:54.631996 11517 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:54.633490 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.621704 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:54.628684 11367 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.554s	user 0.040s	sys 0.043s
I20260709 13:16:54.529567 16796 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.634601 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.635278 16794 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:16:54.636385 16661 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.507s	user 0.013s	sys 0.032s
I20260709 13:16:54.636862 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.637127 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.637694 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.648313 16794 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.648981 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.649503 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.650178 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.650635 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.650748 12063 raft_consensus.cc:2376] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13. Re-sending same reply.
I20260709 13:16:54.648912 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.649884 16707 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.651603 12060 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:54.639417 16467 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 20 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:54.651865 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.651726 16707 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.652674 12063 raft_consensus.cc:2410] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:16:54.653040 12069 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:54.653447 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.652554 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.653808 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.654325 11364 raft_consensus.cc:1813] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Handling vote request from an unknown peer 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:54.654459 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.645735 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.654722 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.653816 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.655119 12061 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:54.665197 11364 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 10: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:54.666822 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.651057 16656 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.669456 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:54.669559 16656 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.670878 11334 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:54.671710 16785 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.673326 16534 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.673679 16654 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:54.674579 16654 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:54.674940 16654 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:54.675855 16654 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.677412 16532 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:54.678134 16755 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.678742 16755 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.680136 16656 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.681018 16755 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.682359 16728 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.682667 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.653811 16707 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.682894 16728 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.683916 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.684934 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:54.685514 16728 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.685972 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:54.686995 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.676874 16654 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:54.688546 16794 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.662752 12033 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:54.690382 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:54.691226 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:54.679006 16785 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:54.705150 16512 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.699913 11439 raft_consensus.cc:2410] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:16:54.704152 11333 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.704849 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.687740 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:54.705696 16728 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.708978 16806 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.709096 16728 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.709790 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.710282 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.710464 11438 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:54.710839 16759 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.711340 16759 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.711654 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.709227 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.713059 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.713315 16759 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.714329 11513 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:16:54.714560 11437 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
I20260709 13:16:54.714897 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.715461 11514 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:54.714350 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.716146 12061 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:54.715377 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.717208 11437 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:54.706426 12063 raft_consensus.cc:2393] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:54.717403 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:54.734680 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.736280 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.736617 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.736812 16512 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.737457 16794 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:54.737821 16794 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:54.738170 16794 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:54.738862 16512 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.739746 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:54.709499 16806 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.743739 11438 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 9.
I20260709 13:16:54.745836 16806 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.712183 16728 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.748773 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.750873 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:54.750103 11333 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.708065 16650 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:54.794513 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.707490 16812 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:54.751689 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 80894.5us
I20260709 13:16:54.706738 16814 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:54.795498 11437 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 10.
I20260709 13:16:54.801787 11334 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:54.796597 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 46558.8us
I20260709 13:16:54.803246 16510 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:54.803458 16665 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:54.747012 16815 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:54.803889 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.805104 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.805612 12066 raft_consensus.cc:2376] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16. Re-sending same reply.
I20260709 13:16:54.806365 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.807179 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.807514 16510 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.803380 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.808059 16510 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:54.803891 16665 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:54.809157 16665 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.810864 12063 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:54.811739 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.812690 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.813592 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.813741 16794 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.807879 11437 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:54.816327 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.817269 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.817801 11437 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:54.797786 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.673869 16534 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.756924 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.665789 16796 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:54.819816 11439 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:16:54.803658 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.820408 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.822211 11437 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:54.802621 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.826475 16796 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.826550 16815 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.827487 16815 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:54.827961 16534 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.752589 16785 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:54.799001 16812 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:54.830358 16812 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:54.831501 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:54.831830 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.833006 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:54.834161 11437 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:54.834247 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.787115 16532 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:54.747561 16092 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.820317 11514 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:16:54.835419 11513 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:16:54.749234 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.820575 11369 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:16:54.838935 11333 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:54.819706 16814 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.846688 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.853222 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.854452 11369 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:16:54.854688 11364 raft_consensus.cc:2410] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:16:54.855504 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:54.859807 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 58119.7us
I20260709 13:16:54.859309 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.860978 11334 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:54.839040 16532 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.847532 16796 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:54.863242 16534 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.869388 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.869445 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.870074 12062 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:54.893628 16510 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.898583 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2811) took 3125 ms. Trace:
W20260709 13:16:54.944155 16766 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.919s	user 0.004s	sys 0.001s
I20260709 13:16:54.867690 16728 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.926261 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.899284 16512 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.337790 16742 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:54.923038 11370 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.813s	user 0.000s	sys 0.024s
W20260709 13:16:54.965165 11370 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.855s	user 0.002s	sys 0.026s
I20260709 13:16:54.920481 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.867986 16806 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:54.945590 16765 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.874s	user 0.001s	sys 0.001s
I20260709 13:16:54.867877 16794 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:54.894235 11437 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:54.940934 16815 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.935889 16665 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:54.928602 16092 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:54.971702 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:54.971817 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.948958 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:54.949330 16792 log.cc:927] Time spent T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.912s	user 0.001s	sys 0.001s
W20260709 13:16:54.950591 16767 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.923s	user 0.003s	sys 0.000s
I20260709 13:16:54.974617 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:54.929103 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.960415 16661 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.857995 16814 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:54.902062 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:54.962217 11443 rpcz_store.cc:276] 0709 13:16:51.772613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.772758 (+   145us) service_pool.cc:224] Handling call
0709 13:16:51.773084 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:51.911394 (+138310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:51.912365 (+   971us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:51.928097 (+ 15732us) log.cc:844] Serialized 26 byte log entry
0709 13:16:51.999232 (+ 71135us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b18001f3900. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:16:52.117239 (+118007us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b18001f3900. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911557 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:16:52.184293 (+ 67054us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:52.184343 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.898193 (+2713850us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b54002ea754. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:54.898241 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.898285 (+    44us) raft_consensus.cc:1685] finished
0709 13:16:54.898296 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.898410 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46974,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":9617,"spinlock_wait_cycles":300521344,"thread_start_us":184852,"threads_started":1,"wal-append.queue_time_us":328870,"wal-append.run_cpu_time_us":5262,"wal-append.run_wall_time_us":1463230}
I20260709 13:16:54.981496 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1296) took 1015 ms. Trace:
I20260709 13:16:54.981899 12068 rpcz_store.cc:276] 0709 13:16:53.966294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.901800 (+935506us) service_pool.cc:224] Handling call
0709 13:16:54.981429 (+ 79629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:54.871110 16532 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:54.859012 16377 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:54.901110 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.871006 16796 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:54.983044 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2639) took 2941 ms. Trace:
I20260709 13:16:54.926658 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1297) took 2736 ms. Trace:
I20260709 13:16:54.980487 11483 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:54.984493 11366 rpcz_store.cc:276] 0709 13:16:52.190589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.925778 (+2735189us) service_pool.cc:224] Handling call
0709 13:16:54.926143 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:54.926345 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.926371 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:54.926407 (+    36us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:54.926433 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.926480 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.926585 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:54.932512 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.847012 16653 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:54.962039 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 211249us
I20260709 13:16:54.969775 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.890429 16785 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:54.994378 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.977325 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:54.980336 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.982951 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.987134 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.891322 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:54.996949 11439 raft_consensus.cc:3060] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:16:54.997335 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:54.996519 11512 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:54.965586 16650 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:16:54.999758 11334 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:54.999902 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:54.980983 16467 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:54.967428 16512 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.005124 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.005049 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:54.967257 16728 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.005394 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.967672 16534 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.006839 11512 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:54.994345 16092 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:54.986191 16796 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:55.002528 16707 raft_consensus.cc:2804] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:16:55.025175 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 126444us
I20260709 13:16:55.014767 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.013109 11443 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:54.995745 16377 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.984241 16755 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.026746 16755 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.028446 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.028730 11437 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:55.017594 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.006106 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.019018 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.000638 16785 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.021445 16782 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:55.022722 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.002897 16709 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:55.024930 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:55.028685 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:54.984266 11367 rpcz_store.cc:276] 0709 13:16:52.041668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.041852 (+   184us) service_pool.cc:224] Handling call
0709 13:16:52.073668 (+ 31816us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:52.074098 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.074126 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:52.074762 (+   636us) log.cc:844] Serialized 52 byte log entry
0709 13:16:54.635089 (+2560327us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:54.635126 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.635197 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.949923 (+314726us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:54.975538 (+ 25615us) raft_consensus.cc:1685] finished
0709 13:16:54.975561 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.982980 (+  7419us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1675,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":450931968,"thread_start_us":2559971,"threads_started":1,"wal-append.queue_time_us":1951935}
I20260709 13:16:55.007179 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2815) took 3229 ms. Trace:
I20260709 13:16:55.030198 11366 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:55.030593 11511 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:16:55.006839 16467 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.029171 16377 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:54.984436 16812 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.033586 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.032888 16534 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.040956 16712 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:55.041180 11512 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:55.101334 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.046325 11368 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:55.046963 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.100852 11439 raft_consensus.cc:2468] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:55.047744 16512 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.049120 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:55.049333 16728 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.050648 16709 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:55.107880 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:55.031672 16831 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:16:55.017671 16656 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:55.051555 11444 rpcz_store.cc:276] 0709 13:16:51.778291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.778413 (+   122us) service_pool.cc:224] Handling call
0709 13:16:51.778761 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:51.998642 (+219881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:51.998671 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.001458 (+  2787us) log.cc:844] Serialized 26 byte log entry
0709 13:16:53.925020 (+1923562us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:53.925059 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.925136 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.944915 (+1019779us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.002216 (+ 57301us) raft_consensus.cc:1685] finished
0709 13:16:55.002238 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.007108 (+  4870us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14379,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":351526656,"thread_start_us":2120490,"threads_started":2,"wal-append.queue_time_us":2058003}
I20260709 13:16:55.016687 16092 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:55.026038 11408 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.957s (SENT) [suppressed 23 similar messages]
I20260709 13:16:55.058193 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.058777 16841 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:55.059350 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.032874 16742 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:55.059504 16803 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.087569 16665 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:55.091552 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.094007 16755 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.129117 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.095956 16815 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.002724 16661 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.133785 16661 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.056718 11443 raft_consensus.cc:2376] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16. Re-sending same reply.
I20260709 13:16:55.134912 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.135584 11443 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:55.136376 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.136439 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2595) took 1046 ms. Trace:
I20260709 13:16:55.136886 11443 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:55.137756 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.138257 11443 raft_consensus.cc:2376] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11. Re-sending same reply.
I20260709 13:16:55.139060 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.047232 16782 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.102406 11517 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:16:55.194386 11443 raft_consensus.cc:2410] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:16:55.195092 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55430.7us
W20260709 13:16:55.102499 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 96472.4us
W20260709 13:16:55.102751 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 99966.5us
I20260709 13:16:55.056742 16467 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:55.200505 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 200812us
I20260709 13:16:55.062539 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.068596 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:55.106550 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 83896.8us
I20260709 13:16:55.107089 16649 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.108054 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1296) took 2917 ms. Trace:
I20260709 13:16:55.108234 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:54.984124 16806 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.207501 16649 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.209565 16649 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.109617 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:55.209559 16796 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.114634 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.215659 16532 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.216353 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.216951 11438 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:55.217589 16532 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.217721 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.218348 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.218755 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.219435 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.220454 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.221343 16806 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.114902 11516 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:55.223901 11516 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:55.224963 11516 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
W20260709 13:16:55.120357 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 94449.5us
I20260709 13:16:55.225164 16814 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.226284 12066 raft_consensus.cc:2393] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:55.121145 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:55.227988 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 106946us
I20260709 13:16:55.229358 16812 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.111330 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.229992 16812 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:55.230371 11445 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:55.231488 15698 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:55.233700 15698 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:55.235239 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.228029 12066 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:55.235522 16796 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.242842 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.244383 16708 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:55.245368 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 135823us
I20260709 13:16:55.250337 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.251158 16806 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:55.253320 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.254153 11443 raft_consensus.cc:2393] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 19: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:55.230367 16812 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:55.255309 16812 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:55.255656 16650 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:55.256125 16650 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:55.256511 16650 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:55.257342 16742 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:55.258304 16654 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:16:55.259441 11408 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.264246 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.265028 11438 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:55.265986 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.269731 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.261842 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.060772 16789 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.126416 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.277290 11444 raft_consensus.cc:3060] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:55.127628 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.053090 16707 raft_consensus.cc:697] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:55.128244 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.129870 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.281751 11365 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.131166 11366 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 20.
I20260709 13:16:55.110343 16709 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:55.139619 16336 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 3.043s	user 0.023s	sys 0.021s
I20260709 13:16:55.140473 12064 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:55.119757 16656 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:55.136837 11518 rpcz_store.cc:276] 0709 13:16:54.089659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.135570 (+1045911us) service_pool.cc:224] Handling call
0709 13:16:55.135884 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.136056 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.136079 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.136118 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:55.136153 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.136199 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.136290 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.142145 16728 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.153362 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.122113 16092 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:55.158667 16779 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.091s	user 0.000s	sys 0.003s
I20260709 13:16:55.158366 16841 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:55.288614 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:55.162003 16250 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.917s	user 0.012s	sys 0.041s
I20260709 13:16:55.291203 16841 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:16:55.291975 16250 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.047s	user 0.012s	sys 0.042s
I20260709 13:16:55.162458 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.292132 16841 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:55.292968 16841 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.294140 16841 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.166332 16377 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.045637 16785 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.296931 16377 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.297806 11367 raft_consensus.cc:2393] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
W20260709 13:16:55.298094 16788 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 1.157s	user 0.003s	sys 0.000s
I20260709 13:16:55.196750 11333 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.198752 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:16:55.299901 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 101184us
I20260709 13:16:55.200011 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.202404 11515 raft_consensus.cc:2368] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:55.202723 11369 rpcz_store.cc:276] 0709 13:16:52.190309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.870165 (+2679856us) service_pool.cc:224] Handling call
0709 13:16:54.944897 (+ 74732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.945324 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.945351 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.945837 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.107753 (+161916us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:55.107787 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.107848 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.107889 (+    41us) raft_consensus.cc:1685] finished
0709 13:16:55.107899 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.108003 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9830,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":3876,"thread_start_us":161569,"threads_started":1,"wal-append.queue_time_us":159887}
I20260709 13:16:55.199954 16755 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.205732 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:55.301594 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 104924us
I20260709 13:16:55.271819 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:55.276078 11516 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:55.291491 16814 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:55.302464 15698 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:55.302822 15698 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.303108 15698 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:55.303252 11516 raft_consensus.cc:1240] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:55.052657 16797 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:55.278049 16649 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:55.277190 16654 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:55.284559 11438 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:55.284319 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.285352 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.287194 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.259284 16742 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:16:55.290861 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.291260 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.291452 16837 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.278182 16806 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 13:16:55.301404 16336 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.205s	user 0.023s	sys 0.022s
I20260709 13:16:55.195060 16712 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:55.301608 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.317173 11369 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.300868 11444 raft_consensus.cc:2468] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 20.
I20260709 13:16:55.318461 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.300539 12034 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:55.305491 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.159889 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.307209 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.286408 16656 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.310328 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.328168 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.329335 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.329810 11445 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:55.291702 16728 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.306041 16815 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:55.315325 16837 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.333834 16837 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:55.336423 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2647) took 2590 ms. Trace:
I20260709 13:16:55.336987 11439 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:55.336824 11366 rpcz_store.cc:276] 0709 13:16:52.745948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.315198 (+2569250us) service_pool.cc:224] Handling call
0709 13:16:55.335964 (+ 20766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.336130 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.336154 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.336195 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:55.336223 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.336271 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.336370 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.328243 11367 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.312021 11510 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:55.309599 11518 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:16:55.321142 11438 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:55.321859 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.323176 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:55.318975 16789 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:55.298962 16707 consensus_queue.cc:237] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.340809 12063 raft_consensus.cc:2368] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
I20260709 13:16:55.338425 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.340896 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.344761 11444 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:55.308982 16649 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:55.345412 11438 raft_consensus.cc:2376] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14. Re-sending same reply.
I20260709 13:16:55.308739 16654 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:55.348734 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.343283 11369 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:55.350952 11408 leader_election.cc:400] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:55.351275 11408 leader_election.cc:403] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:55.301952 16755 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.335049 16709 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.377167 11438 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:55.388204 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.388844 11438 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:16:55.389479 11439 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:55.350656 16649 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.375568 11369 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:55.392558 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.377641 11444 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:16:55.315012 16534 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:16:55.394541 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2648) took 2648 ms. Trace:
I20260709 13:16:55.351934 16837 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:55.353380 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:55.352460 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47048.6us
I20260709 13:16:55.394902 11369 rpcz_store.cc:276] 0709 13:16:52.746173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.350901 (+2604728us) service_pool.cc:224] Handling call
0709 13:16:55.351176 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.394305 (+ 43129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.394487 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.407994 11366 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:55.396530 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.412248 11444 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:55.378721 16797 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:55.412277 12033 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:55.395622 16377 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:55.415187 16534 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:55.415592 16534 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:55.416308 16534 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.417218 16782 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.417407 16534 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.424118 16838 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:55.424603 16838 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.424933 16838 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:55.352006 16512 consensus_queue.cc:1059] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 13:16:55.426033 16795 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.305s	user 0.000s	sys 0.004s
W20260709 13:16:55.427544 16768 log.cc:927] Time spent T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.390s	user 0.005s	sys 0.000s
I20260709 13:16:55.429140 16828 raft_consensus.cc:3055] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:55.429543 16828 raft_consensus.cc:740] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:55.430547 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2644) took 3367 ms. Trace:
I20260709 13:16:54.097357 11362 rpcz_store.cc:276] 0709 13:16:52.102792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.102964 (+   172us) service_pool.cc:224] Handling call
0709 13:16:52.116032 (+ 13068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.116286 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:52.116626 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.075352 (+1958726us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.096544 (+ 21192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:54.096602 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:54.096643 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.096702 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.096810 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36037,"prepare.queue_time_us":17719,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":106,"raft.queue_time_us":170176,"thread_start_us":1937128,"threads_started":2,"wal-append.queue_time_us":1950170}
I20260709 13:16:55.430922 11361 rpcz_store.cc:276] 0709 13:16:52.063015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.063149 (+   134us) service_pool.cc:224] Handling call
0709 13:16:52.082413 (+ 19264us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:52.082797 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.082823 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:52.083425 (+   602us) log.cc:844] Serialized 52 byte log entry
0709 13:16:54.219002 (+2135577us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:54.219300 (+   298us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.226359 (+  7059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.226439 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.426918 (+1200479us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.430213 (+  3295us) raft_consensus.cc:1685] finished
0709 13:16:55.430315 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.430486 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32955,"prepare.queue_time_us":319,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":23936,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":46389,"spinlock_wait_cycles":426443136,"thread_start_us":2135219,"threads_started":1,"wal-append.queue_time_us":1954144}
I20260709 13:16:55.432475 16709 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:55.433279 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2662) took 1062 ms. Trace:
I20260709 13:16:55.433964 16843 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.434535 16843 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.434762 16828 consensus_queue.cc:260] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.433629 11362 rpcz_store.cc:276] 0709 13:16:54.371063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.432485 (+1061422us) service_pool.cc:224] Handling call
0709 13:16:55.432810 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.432973 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.432996 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.433034 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:55.433068 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.433128 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.433227 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:55.436282 16824 log.cc:927] Time spent T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.405s	user 0.004s	sys 0.000s
I20260709 13:16:55.437213 11362 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:16:55.438627 16828 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:55.444954 16775 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.411s	user 0.003s	sys 0.003s
W20260709 13:16:55.447232 16773 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 1.413s	user 0.005s	sys 0.000s
I20260709 13:16:55.449362 16805 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:55.449816 16805 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.449976 16815 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:55.450776 11362 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:55.452070 11519 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2589) took 3318 ms. Trace:
I20260709 13:16:55.453178 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:55.453469 16772 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.427s	user 0.002s	sys 0.002s
I20260709 13:16:55.453877 11444 raft_consensus.cc:2368] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 19. Current term is 20.
I20260709 13:16:55.414750 11366 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:55.456902 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1301) took 1339 ms. Trace:
W20260709 13:16:55.457388 11442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.401s	user 0.011s	sys 0.054s
W20260709 13:16:55.457710 11442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.406s	user 0.011s	sys 0.054s
I20260709 13:16:55.458401 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2813) took 3682 ms. Trace:
I20260709 13:16:55.459602 16756 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:55.460036 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1298) took 3220 ms. Trace:
I20260709 13:16:55.460188 16756 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.460440 11368 rpcz_store.cc:276] 0709 13:16:52.239852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.115978 (+2876126us) service_pool.cc:224] Handling call
0709 13:16:55.116279 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.130684 (+ 14405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.130715 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.206295 (+ 75580us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.316674 (+110379us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:55.316712 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.316776 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.458556 (+141780us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.459760 (+  1204us) raft_consensus.cc:1685] finished
0709 13:16:55.459781 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.459977 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7551,"raft.queue_time_us":71952,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":10277,"spinlock_wait_cycles":77491584,"thread_start_us":110020,"threads_started":1,"wal-append.queue_time_us":217231}
I20260709 13:16:55.462500 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2671) took 3511 ms. Trace:
I20260709 13:16:55.464490 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.459770 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2641) took 3404 ms. Trace:
W20260709 13:16:55.467623 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 162168us
I20260709 13:16:55.457201 11366 rpcz_store.cc:276] 0709 13:16:54.117462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.413571 (+1296109us) service_pool.cc:224] Handling call
0709 13:16:55.414575 (+  1004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.456697 (+ 42122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.456852 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.452440 11519 rpcz_store.cc:276] 0709 13:16:52.133230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.140948 (+  7718us) service_pool.cc:224] Handling call
0709 13:16:52.141294 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.141701 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.141731 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.142256 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.039519 (+1897263us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:54.039559 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.039626 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.446474 (+1406848us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.451754 (+  5280us) raft_consensus.cc:1685] finished
0709 13:16:55.451777 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.452011 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20582,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"spinlock_wait_cycles":435712256,"thread_start_us":1896874,"threads_started":1,"wal-append.queue_time_us":1890997}
I20260709 13:16:55.390645 16759 raft_consensus.cc:3060] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.385479 16796 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:55.461844 11484 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.350452 16654 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.463078 11441 rpcz_store.cc:276] 0709 13:16:51.951036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.088124 (+137088us) service_pool.cc:224] Handling call
0709 13:16:52.088421 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.088790 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.088815 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.089278 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:16:53.975662 (+1886384us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:53.975713 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.975792 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.444220 (+1468428us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.462209 (+ 17989us) raft_consensus.cc:1685] finished
0709 13:16:55.462232 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.462430 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6066,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":435007232,"thread_start_us":1885999,"threads_started":1,"wal-append.queue_time_us":1944564}
I20260709 13:16:55.473441 11510 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:55.477027 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.477604 11511 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:55.479646 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.474188 11443 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:16:55.469611 11335 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.483052 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.465471 11370 rpcz_store.cc:276] 0709 13:16:52.055096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.055484 (+   388us) service_pool.cc:224] Handling call
0709 13:16:52.069244 (+ 13760us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:52.075368 (+  6124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.098791 (+ 23423us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:52.110049 (+ 11258us) log.cc:844] Serialized 52 byte log entry
0709 13:16:54.965538 (+2855489us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:55.021114 (+ 55576us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.021318 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.025037 (+  3719us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.452811 (+427774us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.459460 (+  6649us) raft_consensus.cc:1685] finished
0709 13:16:55.459566 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.459718 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":55582,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":76035,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":148,"spinlock_wait_cycles":451234688,"thread_start_us":2905452,"threads_started":2,"wal-append.queue_time_us":1916049}
I20260709 13:16:55.450142 16805 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:55.491043 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.494352 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.495636 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.395385 16728 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.496304 11517 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:55.496771 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.498000 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.500655 16467 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.502861 16467 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.503916 11513 raft_consensus.cc:2393] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:55.506270 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.476981 16843 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.479101 16812 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.508844 16787 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 19 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.509693 16812 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:16:55.509758 11512 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:55.511098 11513 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:55.513682 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:55.514854 11335 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.458734 11442 rpcz_store.cc:276] 0709 13:16:51.775611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.779684 (+  4073us) service_pool.cc:224] Handling call
0709 13:16:51.779947 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:51.997085 (+217138us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:51.997118 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.010883 (+ 13765us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.943614 (+2932731us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.458024 (+514410us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:55.458066 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.458133 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.458174 (+    41us) raft_consensus.cc:1685] finished
0709 13:16:55.458185 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.458357 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13387,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":76,"spinlock_wait_cycles":454941568,"thread_start_us":3623032,"threads_started":2,"wal-append.queue_time_us":2002373}
I20260709 13:16:55.517150 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.520680 11511 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:55.512694 15698 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.522604 11514 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15. Ops: [13.8-14.9]
I20260709 13:16:55.524278 11445 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:55.480998 16806 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:55.526870 11517 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:55.527209 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.485435 16742 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:55.527957 11441 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.528875 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.535451 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.492479 16653 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:55.536948 15698 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.476971 16774 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.536613 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:55.538337 11335 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.542804 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.551702 11335 leader_election.cc:400] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 19 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 19. Current term is 20.
I20260709 13:16:55.551985 12060 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:16:55.552064 11335 leader_election.cc:403] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:16:55.522349 16712 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:55.558241 16797 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.551079 16630 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:55.563355 16630 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.563738 16630 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:55.525791 16467 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:55.564167 16467 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:55.564638 16755 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.522148 16796 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:55.566053 16796 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.566246 16661 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:55.500213 16815 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:16:55.566540 16650 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.529445 16806 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.568296 16806 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:55.568683 16650 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:55.573490 16784 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 1.546s	user 0.002s	sys 0.003s
I20260709 13:16:55.574095 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2646) took 2828 ms. Trace:
I20260709 13:16:55.573078 16092 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.522271 16843 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:55.522198 16794 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:55.564538 16467 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:55.592911 16844 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:55.593554 16844 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.593891 16844 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:55.594233 16785 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.595582 11364 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 14: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:55.496330 16837 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:16:55.536739 16803 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:55.577527 11365 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:55.577345 11367 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:55.556099 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.598094 11445 raft_consensus.cc:2410] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 16 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:16:55.588397 16656 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.589464 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.589859 16794 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.590466 16759 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:55.579814 16774 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 19. Current term is 20.
I20260709 13:16:55.591765 16712 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:55.595480 12068 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 17: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:55.525835 16787 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:55.536597 16653 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.607193 16803 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:55.609458 11513 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:16:55.574502 11363 rpcz_store.cc:276] 0709 13:16:52.745651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.154677 (+2409026us) service_pool.cc:224] Handling call
0709 13:16:55.155033 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.200552 (+ 45519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.200593 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.219789 (+ 19196us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.350292 (+130503us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:55.350331 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.350394 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.573860 (+223466us) raft_consensus.cc:1685] finished
0709 13:16:55.573880 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.574033 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45079,"raft.queue_time_us":45596,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":4406,"spinlock_wait_cycles":677632,"thread_start_us":130158,"threads_started":1,"wal-append.queue_time_us":192325}
I20260709 13:16:55.624771 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.628007 11519 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:55.629644 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.629799 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:55.630396 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.611135 16794 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:55.631006 16661 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:55.643303 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.644018 11445 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:55.627040 16787 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:55.627283 16653 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:55.648067 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 110720us
I20260709 13:16:55.645026 11333 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:55.651458 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 123997us
I20260709 13:16:55.653136 11484 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.656080 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.659797 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.673237 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.674306 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.674935 11517 raft_consensus.cc:2368] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 19. Current term is 20.
I20260709 13:16:55.640672 16661 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:55.649485 16787 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:55.676105 16661 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.677160 16661 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:55.647923 16785 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.656596 16843 raft_consensus.cc:2764] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:16:55.684551 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:55.691316 16787 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.694484 16787 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.698627 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.699517 11445 raft_consensus.cc:2368] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 19. Current term is 20.
I20260709 13:16:55.700448 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.701643 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.702275 11445 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:16:55.703578 16532 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:55.704109 16532 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:55.704483 16532 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:55.711841 16649 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.714403 16649 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.715740 11441 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:55.716552 16654 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.718811 16654 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.703110 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.720379 11445 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:55.720920 11519 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
W20260709 13:16:55.721946 16783 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.683s	user 0.007s	sys 0.005s
I20260709 13:16:55.722967 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.723271 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2673) took 3725 ms. Trace:
I20260709 13:16:55.724077 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.724834 11519 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 10: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:55.716737 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.730347 11335 leader_election.cc:400] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 19. Current term is 20.
I20260709 13:16:55.735527 11442 raft_consensus.cc:2468] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:55.731523 11335 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.740226 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.741114 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.742038 16843 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:55.742225 16786 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.742681 16774 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:55.743212 16774 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:55.739523 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:55.745090 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.743651 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:55.747288 12062 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:55.748333 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.723644 11436 rpcz_store.cc:276] 0709 13:16:51.998074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.135428 (+137354us) service_pool.cc:224] Handling call
0709 13:16:52.186543 (+ 51115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.231548 (+ 45005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.231578 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.242485 (+ 10907us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.612768 (+2370283us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:54.612808 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.612867 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.681364 (+1068497us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.722964 (+ 41600us) raft_consensus.cc:1685] finished
0709 13:16:55.722987 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.723212 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426746624,"thread_start_us":2330611,"threads_started":1,"wal-append.queue_time_us":1756299}
W20260709 13:16:55.749079 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 96043.2us
I20260709 13:16:55.749835 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.757854 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.751793 11335 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.743611 16774 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:55.761565 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:16:55.762349 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.751422 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.765540 11519 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:55.765741 11484 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.762893 16787 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:55.719810 16806 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.759095 16578 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:55.783229 16578 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.783731 16578 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:55.786197 16756 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.788472 16793 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.743907 16843 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.739086 16532 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.742865 16786 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:55.775933 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.776146 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.797998 11436 raft_consensus.cc:2368] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:16:55.769559 16661 raft_consensus.cc:2764] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:16:55.776988 16858 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:55.776996 16860 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:55.778667 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.801204 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.802361 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1160) took 1117 ms. Trace:
I20260709 13:16:55.802660 12062 raft_consensus.cc:2468] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:55.802810 11444 rpcz_store.cc:276] 0709 13:16:54.684649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.454414 (+769765us) service_pool.cc:224] Handling call
0709 13:16:55.454667 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.454845 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:55.662751 (+207906us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.801815 (+139064us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.801984 (+   169us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:55.802043 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.802130 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.802305 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":288550,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":89,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":306774,"threads_started":2,"wal-append.queue_time_us":138719}
I20260709 13:16:55.804456 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:55.780361 16656 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.782160 16806 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.760353 16803 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.760784 16785 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:16:55.790493 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:55.802824 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:55.813452 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.805023 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:55.815737 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.815969 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.799446 16843 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:55.816553 11436 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:55.815212 11444 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:55.817793 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.821007 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.821991 16860 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:55.822700 16860 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:55.823962 16794 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.825294 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.826529 16846 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:55.826931 16846 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.827277 16846 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:55.827816 16847 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.408s	user 0.003s	sys 0.000s
I20260709 13:16:55.827940 16630 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.828485 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.829892 11483 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:55.830077 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1163) took 1144 ms. Trace:
I20260709 13:16:55.830637 16510 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.831588 12065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1483) took 1306 ms. Trace:
W20260709 13:16:55.833038 16799 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 1.710s	user 0.004s	sys 0.000s
I20260709 13:16:55.831956 12065 rpcz_store.cc:276] 0709 13:16:54.525597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.529053 (+  3456us) service_pool.cc:224] Handling call
0709 13:16:54.559285 (+ 30232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:54.559521 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:54.575764 (+ 16243us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.435894 (+860130us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b2400311be8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b97979c4 00007f35b9795acb 00007f35b9950dc9 00007f35b9950a55 00007f35b995009d 00007f35bc58210e 00007f35bc59988c 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221
0709 13:16:55.831050 (+395156us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.831232 (+   182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.831280 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:55.831329 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.831416 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.831520 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":879202,"prepare.queue_time_us":896046,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":325,"raft.queue_time_us":23433,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":51,"spinlock_wait_cycles":29983616,"thread_start_us":375549,"threads_started":1,"wal-append.queue_time_us":453298}
I20260709 13:16:55.833200 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:55.834357 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:55.808776 16650 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:55.835763 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:55.837182 11365 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:55.837320 11335 leader_election.cc:400] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:16:55.838809 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:55.830443 11441 rpcz_store.cc:276] 0709 13:16:54.685419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.829238 (+1143819us) service_pool.cc:224] Handling call
0709 13:16:55.829536 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.829733 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.829757 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.829815 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:55.829861 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.829916 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.830009 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.857842 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1162) took 1172 ms. Trace:
I20260709 13:16:55.858373 11444 rpcz_store.cc:276] 0709 13:16:54.685154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.819269 (+1134115us) service_pool.cc:224] Handling call
0709 13:16:55.857289 (+ 38020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.857476 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.857503 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.857546 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:55.857578 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.857647 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.857771 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.859349 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.809875 16785 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.869920 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:55.817586 16661 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:55.819664 16858 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:55.877579 16832 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:16:55.880626 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 104780us
I20260709 13:16:55.881388 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:55.821059 16800 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.855744 16630 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.855993 16650 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:55.864966 16510 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:55.884996 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.888692 11441 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:55.880216 15698 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:55.870985 16656 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:55.864799 16786 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:55.878752 16858 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.873648 16785 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:55.899829 11484 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:55.904883 11299 debug-util.cc:398] Leaking SignalData structure 0x7b0800035440 after lost signal to thread 11302
I20260709 13:16:55.906163 11436 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
W20260709 13:16:55.906390 11299 debug-util.cc:398] Leaking SignalData structure 0x7b08002c7fe0 after lost signal to thread 11394
I20260709 13:16:55.864742 16844 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:55.908133 11299 debug-util.cc:398] Leaking SignalData structure 0x7b080040b540 after lost signal to thread 11469
W20260709 13:16:55.908553 11299 debug-util.cc:398] Leaking SignalData structure 0x7b080038ee00 after lost signal to thread 11470
I20260709 13:16:55.888265 16650 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:55.910552 11299 debug-util.cc:398] Leaking SignalData structure 0x7b080035bca0 after lost signal to thread 11543
I20260709 13:16:55.897801 16661 raft_consensus.cc:2804] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:16:55.915036 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 85209.2us
W20260709 13:16:55.913995 11299 debug-util.cc:398] Leaking SignalData structure 0x7b08001c6e80 after lost signal to thread 12093
W20260709 13:16:55.933670 11299 debug-util.cc:398] Leaking SignalData structure 0x7b0800347ec0 after lost signal to thread 12094
W20260709 13:16:55.934141 11299 debug-util.cc:398] Leaking SignalData structure 0x7b080032a200 after lost signal to thread 16410
I20260709 13:16:55.920569 11335 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.923594 11483 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:55.936932 11368 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:55.936878 11443 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
W20260709 13:16:55.939940 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 60088.3us
I20260709 13:16:55.940160 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1164) took 1254 ms. Trace:
I20260709 13:16:55.945700 11443 rpcz_store.cc:276] 0709 13:16:54.685674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.939182 (+1253508us) service_pool.cc:224] Handling call
0709 13:16:55.939533 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.939764 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.939792 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.939838 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:55.939893 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.939954 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.940084 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:55.947558 11299 debug-util.cc:398] Leaking SignalData structure 0x7b08003b39a0 after lost signal to thread 16718
W20260709 13:16:55.948110 11299 debug-util.cc:398] Leaking SignalData structure 0x7b08003e4280 after lost signal to thread 16731
W20260709 13:16:55.948429 11299 debug-util.cc:398] Leaking SignalData structure 0x7b08002e23a0 after lost signal to thread 16737
I20260709 13:16:55.906343 16794 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:55.941978 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:55.959383 12067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1317) took 1313 ms. Trace:
I20260709 13:16:55.959825 12067 rpcz_store.cc:276] 0709 13:16:54.646295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.898169 (+251874us) service_pool.cc:224] Handling call
0709 13:16:54.898444 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.018149 (+119705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.018177 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.099522 (+ 81345us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.289746 (+190224us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:55.289786 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.289856 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.827144 (+537288us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:55.936992 (+109848us) raft_consensus.cc:1685] finished
0709 13:16:55.937014 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.959317 (+ 22303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26192,"raft.queue_time_us":76030,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":80509696,"thread_start_us":271460,"threads_started":2,"wal-append.queue_time_us":282544}
I20260709 13:16:55.905539 16858 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:55.967049 16862 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:16:55.967872 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:55.968465 11436 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:55.969997 16860 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:16:55.971915 16857 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:16:55.972180 16532 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:55.965253 16510 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:16:55.975489 16854 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:55.975816 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1166) took 1289 ms. Trace:
I20260709 13:16:55.980335 11441 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:55.981233 11443 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:55.983875 11441 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:16:55.984756 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1167) took 1298 ms. Trace:
I20260709 13:16:55.985208 11441 rpcz_store.cc:276] 0709 13:16:54.686479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.983062 (+1296583us) service_pool.cc:224] Handling call
0709 13:16:55.983462 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.984463 (+  1001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.984689 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.976263 11439 rpcz_store.cc:276] 0709 13:16:54.686218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.974971 (+1288753us) service_pool.cc:224] Handling call
0709 13:16:55.975300 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.975501 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.975527 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:55.975590 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:55.975618 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.975672 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.975765 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.971200 16860 raft_consensus.cc:2804] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:55.998517 11436 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:16:55.963302 16656 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:55.962322 16786 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:55.909399 16467 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:55.975391 16510 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:56.014104 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2832) took 1310 ms. Trace:
I20260709 13:16:56.016513 11439 rpcz_store.cc:276] 0709 13:16:54.696687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.996417 (+1299730us) service_pool.cc:224] Handling call
0709 13:16:56.006327 (+  9910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.006540 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.006566 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.006611 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:56.006652 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.006709 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.006848 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.997004 16854 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:55.921161 16805 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:55.922554 16838 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.005242 16656 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.031019 11519 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.014353 16860 raft_consensus.cc:697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:56.036484 16874 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:56.040601 16844 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.041066 16805 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:56.042491 16578 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.043280 16861 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:56.045692 16867 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 13:16:56.089735 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.046594 16864 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:16:56.049146 11335 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:56.050284 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:56.093297 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44264.5us
W20260709 13:16:56.050626 16869 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:16:56.052802 16838 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:56.053340 12064 raft_consensus.cc:1240] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:16:56.018416 16786 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:56.053785 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.057906 16832 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:56.064734 16092 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:56.066835 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.023196 16854 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:56.103143 11442 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:16:56.103955 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 161911us
W20260709 13:16:56.104687 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 141908us
I20260709 13:16:56.068432 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:56.084303 11439 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:16:56.014045 16414 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:16:56.032475 16656 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:56.093983 11335 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:56.125453 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 57095.2us
I20260709 13:16:56.126216 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:56.126669 11367 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
W20260709 13:16:56.100972 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47282.6us
I20260709 13:16:56.102843 16832 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:56.086853 16661 raft_consensus.cc:697] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:56.131986 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.127115 11333 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:16:56.098240 16828 consensus_meta.cc:287] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.659s	user 0.007s	sys 0.007s
I20260709 13:16:56.105389 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:56.122704 15852 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.124276 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2835) took 1426 ms. Trace:
I20260709 13:16:56.099193 16844 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:56.088192 16782 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:16:56.137746 11439 rpcz_store.cc:276] 0709 13:16:54.697431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.032092 (+1334661us) service_pool.cc:224] Handling call
0709 13:16:56.084029 (+ 51937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.123965 (+ 39936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.124220 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.141971 11333 leader_election.cc:400] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 19 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 19. Current term is 20.
I20260709 13:16:56.090763 16653 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:56.144038 16771 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 2.076s	user 0.004s	sys 0.001s
I20260709 13:16:56.135131 16532 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:56.102495 16786 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.144493 16811 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:56.168308 16811 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:56.168612 16871 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:16:56.130656 16794 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:16:56.169656 16870 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:16:56.170262 16661 consensus_queue.cc:237] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.134711 16803 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:56.171614 16467 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:56.138304 16510 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.172734 16250 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:56.173106 16250 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:56.173856 16250 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.174888 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54376 (request call id 1498) took 1056 ms. Trace:
I20260709 13:16:56.175372 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1177) took 1456 ms. Trace:
I20260709 13:16:56.175746 11512 rpcz_store.cc:276] 0709 13:16:54.719146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.108787 (+389641us) service_pool.cc:224] Handling call
0709 13:16:55.509593 (+400806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.510846 (+  1253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.175199 (+664353us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54000672d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:56.175319 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921124608}
I20260709 13:16:56.177073 11512 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.179241 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.180001 11362 raft_consensus.cc:2368] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:56.179747 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.180379 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.180776 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.183295 16806 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:56.183856 16806 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.185865 16806 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.186144 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.145836 16887 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:56.186710 11439 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:56.175243 12068 rpcz_store.cc:276] 0709 13:16:55.118335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.118451 (+   116us) service_pool.cc:224] Handling call
0709 13:16:56.174716 (+1056265us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400509954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:56.174834 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727082752}
I20260709 13:16:56.102494 16860 consensus_queue.cc:237] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.106330 16838 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:56.140336 12033 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:56.179804 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.195030 11440 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:56.149163 11483 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:56.149361 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.156142 16532 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:56.157272 11370 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:56.159760 11442 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:16:56.157145 16653 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.160399 16578 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.161046 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2853) took 1214 ms. Trace:
W20260709 13:16:56.142159 16878 log.cc:927] Time spent T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:56.203286 11437 rpcz_store.cc:276] 0709 13:16:54.946737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.004873 (+ 58136us) service_pool.cc:224] Handling call
0709 13:16:56.160861 (+1155988us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b5400438fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:56.160968 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34858240}
I20260709 13:16:56.162393 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.162540 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.166589 11518 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
W20260709 13:16:56.197233 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48243.6us
I20260709 13:16:56.197510 12061 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:56.197052 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 56794.8us
I20260709 13:16:56.199723 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.174806 16250 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.147148 16774 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.187327 16887 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:56.318533 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 63143.3us
W20260709 13:16:56.212940 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.212834 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:56.226342 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.190690 16844 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:56.328347 11437 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:56.329046 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1173) took 1613 ms. Trace:
I20260709 13:16:56.329365 11437 rpcz_store.cc:276] 0709 13:16:54.716100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.327831 (+1611731us) service_pool.cc:224] Handling call
0709 13:16:56.328167 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.328809 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.328992 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.330993 11437 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:56.312290 11439 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:16:56.331542 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1175) took 1615 ms. Trace:
I20260709 13:16:56.331873 11437 rpcz_store.cc:276] 0709 13:16:54.716653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.330555 (+1613902us) service_pool.cc:224] Handling call
0709 13:16:56.330839 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.331316 (+   477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.331493 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.295236 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1171) took 1579 ms. Trace:
I20260709 13:16:56.332584 11442 rpcz_store.cc:276] 0709 13:16:54.715468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.202900 (+1487432us) service_pool.cc:224] Handling call
0709 13:16:56.203314 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.294801 (+ 91487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.294838 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.294894 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:56.294928 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.295005 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.295165 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.334482 11439 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:56.201258 16814 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:56.337249 11437 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:56.338537 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:56.339182 11362 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:56.336720 11409 connection.cc:256] RPC call timeout handler was delayed by -1.3036456770000768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28540000000000004s for a response
W20260709 13:16:56.340794 11409 connection.cc:256] RPC call timeout handler was delayed by -1.3036456770000768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28540000000000004s for a response
W20260709 13:16:56.341183 11409 connection.cc:256] RPC call timeout handler was delayed by -1.1928433000000496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3351s for a response
W20260709 13:16:56.341521 11409 connection.cc:256] RPC call timeout handler was delayed by -1.1928433000000496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3351s for a response
I20260709 13:16:56.340688 11366 raft_consensus.cc:2393] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:56.351202 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2685) took 1047 ms. Trace:
I20260709 13:16:56.351697 11437 rpcz_store.cc:276] 0709 13:16:55.290062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.336779 (+1046717us) service_pool.cc:224] Handling call
0709 13:16:56.337053 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.337661 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.337833 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.353111 11437 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:16:56.353449 11437 pending_rounds.cc:85] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 9
I20260709 13:16:56.353647 11437 pending_rounds.cc:107] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:16:56.353852 11437 raft_consensus.cc:2889] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:56.354497 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1177) took 1032 ms. Trace:
I20260709 13:16:56.354784 11437 rpcz_store.cc:276] 0709 13:16:55.322584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.352445 (+1029861us) service_pool.cc:224] Handling call
0709 13:16:56.352761 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.354267 (+  1506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.354449 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.205088 16794 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:56.318351 16887 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:56.201395 16510 consensus_queue.cc:1059] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 13:16:56.321377 16756 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.364730 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2848) took 1516 ms. Trace:
W20260709 13:16:56.369119 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.365190 11439 rpcz_store.cc:276] 0709 13:16:54.819153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.331781 (+1512628us) service_pool.cc:224] Handling call
0709 13:16:56.332053 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.334970 (+  2917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.335187 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:56.323160 16797 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:16:56.319993 16650 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.356693 16709 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:56.379019 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 479207us
I20260709 13:16:56.381444 11439 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.336328 16578 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:16:56.358114 16814 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:56.386026 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:56.399891 11439 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:56.401491 11439 raft_consensus.cc:1240] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.364876 16858 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:56.384397 16707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.010s	sys 0.005s
W20260709 13:16:56.409446 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 64902us
I20260709 13:16:56.415273 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:56.399907 16793 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:56.374835 16756 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.418430 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:56.420470 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.413283 16707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.010s	sys 0.005s
I20260709 13:16:56.425045 11510 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 17: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:56.391217 16709 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.418879 16883 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:56.421736 16884 raft_consensus.cc:2804] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:56.420178 16793 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:56.390774 16532 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.401021 16844 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.402379 16811 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.432130 16883 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:16:56.454906 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45669.2us
I20260709 13:16:56.414047 16887 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.460866 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.465307 11439 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:56.446759 16843 consensus_meta.cc:287] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.581s	user 0.006s	sys 0.004s
I20260709 13:16:56.419551 16794 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.443562 16884 raft_consensus.cc:697] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:56.446766 16828 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:56.448801 16650 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.440500 16885 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:56.429740 16814 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.449995 16707 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:16:56.478674 16707 raft_consensus.cc:3055] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:56.479038 16707 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:56.480901 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.481082 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.481623 11516 raft_consensus.cc:2368] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:16:56.482252 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.482403 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.483260 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.483695 16885 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:56.484213 11439 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:56.485733 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.485409 16885 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.488269 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.484196 16828 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:56.489753 16894 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:56.491137 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.491639 11439 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:56.492206 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.492472 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.494695 11408 leader_election.cc:400] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:16:56.495297 11408 leader_election.cc:403] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:16:56.495921 11365 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:56.497546 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.498593 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.499907 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.502278 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.502900 11518 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:16:56.503922 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.504923 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.506023 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.506999 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.507179 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2650) took 3760 ms. Trace:
I20260709 13:16:56.507568 11369 rpcz_store.cc:276] 0709 13:16:52.746632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.408944 (+2662312us) service_pool.cc:224] Handling call
0709 13:16:55.409622 (+   678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.486667 (+ 77045us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:56.063199 (+576532us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b18003dffc0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b988bc0a 00007f35c2fcb219 00007f35b9913a40 00007f35b990e4e4 00007f35b990e455 00007f35c03f63c7 00007f35bca089a7 00007f35bca180bf 00007f35bca18038
0709 13:16:56.125411 (+ 62212us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.402014 (+276603us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.408557 (+  6543us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.408710 (+   153us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:56.408915 (+   205us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.411435 (+  2520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.411506 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.506959 (+ 95453us) raft_consensus.cc:1685] finished
0709 13:16:56.506981 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.507120 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71675,"prepare.queue_time_us":580533,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":35182,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":22282,"spinlock_wait_cycles":18787712,"thread_start_us":306083,"threads_started":2,"wal-append.queue_time_us":324329}
I20260709 13:16:56.509660 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:56.510354 11369 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:56.511076 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.493453 16803 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:56.547281 12061 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:56.494234 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1316) took 1657 ms. Trace:
W20260709 13:16:56.494761 16895 log.cc:927] Time spent T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:16:56.511933 16891 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:16:56.512290 16092 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:56.492906 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.514813 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.516053 16861 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:56.521651 11443 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1165) took 1825 ms. Trace:
W20260709 13:16:56.526630 16846 consensus_meta.cc:287] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.699s	user 0.003s	sys 0.006s
W20260709 13:16:56.526774 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 393498us
I20260709 13:16:56.528151 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1318) took 1551 ms. Trace:
I20260709 13:16:56.589097 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.589406 11368 rpcz_store.cc:276] 0709 13:16:54.976369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.537734 (+561365us) service_pool.cc:224] Handling call
0709 13:16:55.555692 (+ 17958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.937931 (+382239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.527883 (+589952us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b540035b454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:56.528084 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876218240}
I20260709 13:16:56.528918 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47990 (request call id 1305) took 2086 ms. Trace:
I20260709 13:16:56.529495 11361 raft_consensus.cc:1240] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.531013 16887 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:56.494247 16890 log.cc:927] Time spent T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:56.540256 16756 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.546818 11362 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:56.461251 16774 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:56.547659 16803 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:56.547956 11370 rpcz_store.cc:276] 0709 13:16:54.836519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.485940 (+649421us) service_pool.cc:224] Handling call
0709 13:16:55.503922 (+ 17982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.198697 (+694775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.494023 (+295326us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b5400249854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:56.494175 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502159744}
I20260709 13:16:56.554996 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.602470 11366 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:56.603664 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2834) took 1906 ms. Trace:
I20260709 13:16:56.586156 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.587468 11443 rpcz_store.cc:276] 0709 13:16:54.685985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.947726 (+1261741us) service_pool.cc:224] Handling call
0709 13:16:55.981031 (+ 33305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.981644 (+   613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.510916 (+529272us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b5400289bd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:56.511058 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75478784}
I20260709 13:16:56.604033 11436 rpcz_store.cc:276] 0709 13:16:54.697188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.999990 (+1302802us) service_pool.cc:224] Handling call
0709 13:16:56.000393 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.014513 (+ 14120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.014550 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.020700 (+  6150us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.153863 (+133163us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:56.153891 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.153937 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.603463 (+449526us) raft_consensus.cc:1685] finished
0709 13:16:56.603484 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.603610 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7640,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6621,"thread_start_us":64496,"threads_started":1,"wal-append.queue_time_us":322758}
I20260709 13:16:56.605384 11369 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:56.605669 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:56.606285 11436 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:56.607577 11443 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:56.472340 16843 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.449683 16854 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.590353 11364 rpcz_store.cc:276] 0709 13:16:54.442298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.859097 (+416799us) service_pool.cc:224] Handling call
0709 13:16:56.528699 (+1669602us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54002dad54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:56.528831 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457616256}
I20260709 13:16:56.610750 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.591447 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:56.592185 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 173831us
I20260709 13:16:56.588320 16092 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:56.590704 16876 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.594509 16846 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.599814 12064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1511) took 1070 ms. Trace:
I20260709 13:16:56.592135 16861 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.596736 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.542462 16832 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.610698 11443 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:16:56.615891 12064 rpcz_store.cc:276] 0709 13:16:55.529394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.579594 (+ 50200us) service_pool.cc:224] Handling call
0709 13:16:55.587928 (+  8334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.100897 (+512969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.599547 (+498650us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54005d6954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:56.599757 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069799296}
W20260709 13:16:56.610244 16785 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.699s	user 0.008s	sys 0.002s
I20260709 13:16:56.621024 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.624363 11362 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:56.628355 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.629048 12061 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:56.629635 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.630152 12068 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:56.630798 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.631343 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.631529 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.547230 16793 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.632932 11514 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.639348 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.642421 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.623819 16817 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:56.643596 16906 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:56.644035 16817 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:56.644167 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.644905 16817 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.645656 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.646294 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.646962 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.647627 12060 raft_consensus.cc:2368] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:56.647855 12064 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:56.648893 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.639462 16884 consensus_queue.cc:237] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.649467 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.649545 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.626891 16785 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.650743 11363 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 9. Current term is 10. Ops: []
I20260709 13:16:56.651324 12060 raft_consensus.cc:2368] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:56.651813 16785 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.643710 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.635708 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.622622 16876 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.654644 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.656323 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.656454 16876 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:56.657229 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:56.658368 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 163804us
I20260709 13:16:56.659659 11409 leader_election.cc:400] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:56.660619 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:16:56.661434 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.624790 16828 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:16:56.662132 11409 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.658568 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.662840 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.663162 12063 raft_consensus.cc:2368] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 16. Current term is 17.
I20260709 13:16:56.664335 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:56.634505 16594 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.647869 16846 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.675832 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:56.678121 11362 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:56.683661 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:56.665643 11408 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:56.688961 11364 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:56.625793 16838 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:16:56.703902 11513 raft_consensus.cc:1240] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:56.640789 16858 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.707568 11365 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.709566 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:56.715209 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 53159.8us
I20260709 13:16:56.716189 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.721215 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.702613 16793 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.721942 11518 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:56.722824 11367 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:56.620649 16883 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.728101 12063 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:16:56.722131 16656 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.002s
I20260709 13:16:56.729920 12032 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:56.714448 16906 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:56.702309 16594 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.714596 16898 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:56.732556 16838 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:56.735119 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 102714us
I20260709 13:16:56.731806 16906 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:56.736699 16906 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.676002 16092 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.741322 16843 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.730285 16828 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:56.743521 16904 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:56.745033 12067 raft_consensus.cc:2410] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:16:56.732030 16883 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:56.737397 16838 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.751302 16092 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:56.763993 11436 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:16:56.731570 16656 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.774326 11408 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.754823 16838 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.782829 16898 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:56.783360 16898 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:56.787909 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:56.790095 11438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1158) took 2105 ms. Trace:
I20260709 13:16:56.790506 11438 rpcz_store.cc:276] 0709 13:16:54.684213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.389427 (+705214us) service_pool.cc:224] Handling call
0709 13:16:56.162977 (+773550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.167059 (+  4082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.167083 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.167546 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.364318 (+196772us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:56.364359 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.364426 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.639698 (+275272us) raft_consensus.cc:1685] finished
0709 13:16:56.639716 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.789841 (+150125us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400438fd0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:56.790018 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778199424,"thread_start_us":196395,"threads_started":1,"wal-append.queue_time_us":254254}
I20260709 13:16:56.795552 16898 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.797853 16898 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:56.798437 16898 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:56.801327 11516 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:56.800694 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:56.805728 11436 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:56.807850 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 91758.9us
I20260709 13:16:56.808642 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.809260 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:56.810448 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:56.806945 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.812007 11443 raft_consensus.cc:3060] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:56.814155 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.816036 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.818477 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.819229 12066 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 17: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:56.800806 12033 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:56.821400 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.821962 11436 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:56.822593 15852 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:56.823025 15852 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:56.825419 11370 raft_consensus.cc:1240] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:56.828594 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.829674 12060 raft_consensus.cc:1240] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:56.776051 16854 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:56.836089 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.830618 12064 raft_consensus.cc:1275] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:56.828384 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.833096 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:56.840612 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 326645us
W20260709 13:16:56.841473 11333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.799782 16832 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:56.849035 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 48313.2us
I20260709 13:16:56.850080 15698 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:56.850564 15698 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.850477 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.851562 12066 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:56.839349 16898 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:56.852378 12063 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.857758 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.855463 16414 raft_consensus.cc:2804] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:56.864521 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.855804 16656 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:56.854981 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.835111 16907 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:56.856818 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:56.860728 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.868781 11436 raft_consensus.cc:2468] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:16:56.868781 12033 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.880146 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:56.880914 11436 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:56.877771 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.820300 16837 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:16:56.884255 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:56.883630 15698 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:56.884838 15852 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:56.835276 16884 consensus_queue.cc:1059] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:16:56.889542 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.891208 11443 raft_consensus.cc:2468] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:56.891093 11369 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:56.849874 16785 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:16:56.893195 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:56.896440 12063 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:16:56.881464 16898 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:56.883816 16414 raft_consensus.cc:697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:56.881819 16832 consensus_queue.cc:1059] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:16:56.905972 11517 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1198) took 1325 ms. Trace:
I20260709 13:16:56.888615 16887 consensus_queue.cc:1048] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:16:56.898172 16837 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:56.899541 16859 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:56.906695 16742 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.343s	user 0.015s	sys 0.006s
I20260709 13:16:56.911260 12064 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:56.922910 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 92530us
W20260709 13:16:56.926219 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.926951 16812 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:16:56.927662 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.906502 12067 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 16. Current term is 17. Ops: []
W20260709 13:16:56.926872 16918 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:16:56.931704 11334 leader_election.cc:400] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11.
I20260709 13:16:56.921865 11517 rpcz_store.cc:276] 0709 13:16:55.580628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.675405 (+ 94777us) service_pool.cc:224] Handling call
0709 13:16:55.682974 (+  7569us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:55.685798 (+  2824us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:55.704085 (+ 18287us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.817573 (+113488us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:55.838016 (+ 20443us) log.cc:844] Serialized 52 byte log entry
0709 13:16:55.838192 (+   176us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:55.838427 (+   235us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.750420 (+911993us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b18002b6c00. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:16:56.898706 (+148286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.898793 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.905766 (+  6973us) raft_consensus.cc:1685] finished
0709 13:16:56.905789 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.905918 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7747,"prepare.queue_time_us":33687,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":3164,"raft.queue_time_us":93760,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":26310,"spinlock_wait_cycles":41113856,"thread_start_us":255468,"threads_started":2,"wal-append.queue_time_us":160667}
W20260709 13:16:56.933754 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 62422.9us
I20260709 13:16:56.934733 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:56.924211 16594 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:56.928957 16742 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.365s	user 0.016s	sys 0.007s
W20260709 13:16:56.942584 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 92987.7us
W20260709 13:16:56.939226 16914 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:16:56.930639 16812 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:56.941257 16886 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:56.941047 16742 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:56.942709 16843 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:56.967752 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 99061.5us
I20260709 13:16:56.967890 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:56.976820 16931 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:56.981670 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:56.982538 16898 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:56.983865 16880 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.882s	user 0.000s	sys 0.004s
I20260709 13:16:56.984997 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:56.985303 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1514) took 1261 ms. Trace:
I20260709 13:16:56.985741 12062 rpcz_store.cc:276] 0709 13:16:55.724345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.813258 (+ 88913us) service_pool.cc:224] Handling call
0709 13:16:55.843951 (+ 30693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.859729 (+ 15778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.859769 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.877140 (+ 17371us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.003205 (+126065us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:56.003282 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.003373 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.983166 (+979793us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:56.984999 (+  1833us) raft_consensus.cc:1685] finished
0709 13:16:56.985021 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.985246 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15271,"raft.queue_time_us":18756,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":8463104,"thread_start_us":93772,"threads_started":1,"wal-append.queue_time_us":212759}
I20260709 13:16:56.975495 16907 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.003123 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.000694 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:57.000906 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:16:57.000208 16924 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:16:56.972617 16832 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:16:56.978674 16414 consensus_queue.cc:237] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:56.969064 16843 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.013497 16742 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:57.010614 16922 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:16:57.002620 16886 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:57.020918 16927 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:16:57.021641 16923 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:57.025274 12061 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:57.027383 16907 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:16:57.028118 16921 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:16:57.046406 16861 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.002851 16898 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:57.064527 15698 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:57.064919 16861 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.065629 11515 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:56.974116 16812 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:57.056324 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 86720.9us
I20260709 13:16:57.070845 11363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1330) took 1210 ms. Trace:
I20260709 13:16:57.071210 11363 rpcz_store.cc:276] 0709 13:16:55.860427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.860576 (+   149us) service_pool.cc:224] Handling call
0709 13:16:55.874887 (+ 14311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.651091 (+776204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.070646 (+419555us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b540035be54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:57.070792 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1783968128}
W20260709 13:16:57.061368 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 58323.8us
I20260709 13:16:57.073551 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:57.075738 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.064941 15698 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:57.013160 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49609.4us
I20260709 13:16:57.079159 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:57.000433 16932 consensus_queue.cc:1059] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:16:57.082638 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.083635 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:57.083775 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:57.084460 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:57.051925 16832 consensus_queue.cc:1243] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 6e87b213a4554d94825aceec352fc094 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:16:57.090698 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.097014 15698 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.098258 15698 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:57.101666 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.102680 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.101608 11443 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:57.102840 11363 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:57.104125 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:57.104427 11443 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:57.107215 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:57.088260 16898 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:57.088250 16854 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:57.087917 16630 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:57.100441 16887 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.118319 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.122133 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:57.122393 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:57.053799 16906 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.123917 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.140663 11363 raft_consensus.cc:2393] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:57.124637 16938 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.094470 16934 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:16:57.087999 16861 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:16:57.142004 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:57.118088 16907 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:57.126991 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:57.120188 16932 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:16:57.148018 11436 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:57.149061 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.149796 16250 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:57.149986 16850 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:57.150468 16861 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:57.171372 16861 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:57.172362 16861 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.151006 16907 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:16:57.173586 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.174331 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.160497 16889 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.722s	user 0.002s	sys 0.000s
I20260709 13:16:57.178896 16901 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.179620 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2831) took 2483 ms. Trace:
I20260709 13:16:57.179508 16901 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.142975 16782 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.132418 16854 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.180917 11439 raft_consensus.cc:3055] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:57.141196 16886 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.163916 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:57.181847 16901 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.183058 11369 raft_consensus.cc:1275] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:16:57.183773 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.184464 11369 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:57.183938 16907 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.184636 12067 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:57.185365 16907 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:57.185786 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:57.173718 16861 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:57.165648 16774 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:16:57.187465 16510 consensus_queue.cc:1059] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:16:57.174578 15852 raft_consensus.cc:3055] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:57.188236 15852 raft_consensus.cc:740] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:57.188364 16510 raft_consensus.cc:3055] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:57.188742 16510 raft_consensus.cc:740] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.189031 15852 consensus_queue.cc:260] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.189451 16510 consensus_queue.cc:260] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.190196 15852 raft_consensus.cc:3060] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:57.190562 16510 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:57.156502 16250 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.169299 16812 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:57.192072 16250 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.193089 16250 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.181314 11439 raft_consensus.cc:740] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:57.134140 16630 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.196285 11439 consensus_queue.cc:260] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.197461 11439 raft_consensus.cc:3060] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:57.198339 16630 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.145396 16844 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:16:57.176605 12068 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:57.180040 11441 rpcz_store.cc:276] 0709 13:16:54.696408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.986525 (+1290117us) service_pool.cc:224] Handling call
0709 13:16:55.986915 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.998237 (+ 11322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.998268 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.003892 (+  5624us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.138734 (+134842us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:56.138774 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.138846 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.159800 (+1020954us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:16:57.179308 (+ 19508us) raft_consensus.cc:1685] finished
0709 13:16:57.179329 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.179556 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3553,"spinlock_wait_cycles":4587904,"thread_start_us":71458,"threads_started":1,"wal-append.queue_time_us":382488}
I20260709 13:16:57.196393 16843 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:57.204505 16832 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:57.200240 16838 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.205418 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.205904 16832 consensus_queue.cc:1059] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:57.206584 12034 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.207475 16942 consensus_queue.cc:1059] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:57.207799 16414 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.208416 16414 raft_consensus.cc:3055] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:57.208741 16414 raft_consensus.cc:740] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.205464 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.209215 11370 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:57.210615 11441 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:16:57.210835 11437 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:57.211181 11437 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:57.211460 16854 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.212203 11519 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:16:57.211884 11437 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.213011 11437 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:57.214203 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2663) took 2809 ms. Trace:
I20260709 13:16:57.214552 16936 raft_consensus.cc:2804] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:57.215073 16936 raft_consensus.cc:697] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:16:57.215767 11440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51474 (request call id 2828) took 2535 ms (client timeout 1958 ms). Trace:
I20260709 13:16:57.199644 16844 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:57.215926 16936 consensus_queue.cc:237] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.207342 16786 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:57.217021 16786 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.216532 16844 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.217826 16786 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.218011 16844 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.218878 16786 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.219250 16844 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:57.217969 16630 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:16:57.214573 11361 rpcz_store.cc:276] 0709 13:16:54.404343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.432744 (+1028401us) service_pool.cc:224] Handling call
0709 13:16:56.529307 (+1096563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.594676 (+ 65369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.214026 (+619350us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002dad50. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:57.214149 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2521134464}
I20260709 13:16:57.217227 16414 consensus_queue.cc:260] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.224889 16414 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:57.225934 11442 pending_rounds.cc:85] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 11
I20260709 13:16:57.226260 11442 pending_rounds.cc:107] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:16:57.226477 11442 raft_consensus.cc:2889] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:57.227175 16630 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:57.227566 16630 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.228329 16630 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.228657 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:57.216560 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.229773 11436 raft_consensus.cc:2410] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:16:57.231462 16650 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.226609 16841 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:57.232403 16841 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.233341 16841 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.234714 16841 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:57.235472 11440 rpcz_store.cc:269] 0709 13:16:54.680710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.757540 (+1076830us) service_pool.cc:224] Handling call
0709 13:16:57.215594 (+1458054us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54003e3ad4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:57.215696 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906942208}
I20260709 13:16:57.236948 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.239646 11438 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:57.214236 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:57.243373 11335 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.246219 11516 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:57.246666 11335 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.246984 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:57.249372 11514 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:16:57.252328 11514 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:57.252282 11484 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.253773 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.253834 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:57.255262 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.257362 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:57.258003 11437 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:16:57.258065 11517 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:57.258476 11437 pending_rounds.cc:85] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 8
W20260709 13:16:57.259096 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44925.3us
I20260709 13:16:57.260067 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:57.259979 11335 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.258754 11437 pending_rounds.cc:107] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:16:57.262730 11515 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:16:57.263851 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:57.261634 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:57.264403 11515 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:57.264995 11437 raft_consensus.cc:2889] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:57.264885 11436 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:57.267103 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 164483us
I20260709 13:16:57.267968 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.268841 11439 raft_consensus.cc:1275] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:57.275012 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:57.277041 11514 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:57.282061 11516 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:57.284516 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:57.284801 11514 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:57.287364 11409 leader_election.cc:400] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:57.287757 11409 leader_election.cc:403] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:16:57.279253 16904 consensus_meta.cc:287] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.528s	user 0.000s	sys 0.010s
I20260709 13:16:57.208843 16838 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:57.273923 16650 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:57.292553 11484 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.283490 16665 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:16:57.291877 15852 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:57.280998 16774 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:57.279836 16832 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:57.274142 16793 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:16:57.307178 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 148841us
I20260709 13:16:57.265475 16854 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.308835 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.309327 11436 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:57.290755 16932 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.302696 16630 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:57.313814 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.307004 16785 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:57.308511 16661 consensus_queue.cc:1059] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:16:57.333782 16661 raft_consensus.cc:3055] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:57.334183 16661 raft_consensus.cc:740] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:57.312700 12034 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:57.304548 16336 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:16:57.336212 16886 raft_consensus.cc:2804] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:57.337973 16886 raft_consensus.cc:697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.338912 16886 consensus_queue.cc:237] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.287045 16883 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:16:57.353227 11519 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:16:57.353822 11514 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:57.316702 11334 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.320056 11484 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.355149 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.320933 15852 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.323715 11369 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:57.326879 16906 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.329828 16837 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.303630 16887 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:16:57.305744 16904 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:57.325413 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.304327 16859 consensus_queue.cc:1059] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:16:57.347698 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:57.348402 16336 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:57.308732 16832 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:57.364141 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 53790.8us
I20260709 13:16:57.364274 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.364704 15852 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.318797 16881 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.369582 11516 raft_consensus.cc:3060] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.372789 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.314833 16850 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 13:16:57.378470 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 53131.4us
I20260709 13:16:57.367738 16812 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:57.407227 16939 log.cc:927] Time spent T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:16:57.408517 16336 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.409132 16844 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:57.402670 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.411736 11484 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.412601 16883 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.413861 16883 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.367854 16800 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:57.371454 16782 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:16:57.421697 15852 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:57.422820 16945 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:57.425752 16092 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:57.426133 16092 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.430632 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:57.335001 16661 consensus_queue.cc:260] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.383504 16467 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.387019 16860 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:57.392565 16935 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:57.371102 16594 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:16:57.392966 16653 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.434264 16774 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.373543 16838 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:16:57.369709 16936 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:16:57.394209 11370 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:57.398524 16854 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.398941 11443 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:16:57.399955 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48448 (request call id 1199) took 1464 ms. Trace:
I20260709 13:16:57.400485 11511 raft_consensus.cc:1240] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:57.449003 11443 pending_rounds.cc:85] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 8
I20260709 13:16:57.449406 11443 pending_rounds.cc:107] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:16:57.449366 11510 rpcz_store.cc:276] 0709 13:16:55.935861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.049958 (+114097us) service_pool.cc:224] Handling call
0709 13:16:57.399770 (+1349812us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54002a80d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:57.399899 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762649344}
I20260709 13:16:57.450248 11443 raft_consensus.cc:2889] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:57.402537 16899 consensus_queue.cc:1059] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:57.403112 11367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 2068) took 1249 ms. Trace:
I20260709 13:16:57.403239 16858 consensus_queue.cc:1048] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:57.399839 16859 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:57.404220 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:57.434477 16860 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:57.465828 11367 rpcz_store.cc:276] 0709 13:16:56.177201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.177850 (+   649us) service_pool.cc:224] Handling call
0709 13:16:56.178124 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.723671 (+545547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.402903 (+679232us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54000c6a54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:57.403057 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252291840}
W20260709 13:16:57.466682 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 67277.3us
I20260709 13:16:57.404992 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:16:57.468430 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 63524.8us
I20260709 13:16:57.374207 16832 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:57.385222 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 68609.4us
I20260709 13:16:57.445268 11516 raft_consensus.cc:2468] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:57.370568 16785 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:16:57.426949 16092 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.401371 16709 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.414986 16786 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:16:57.434791 16920 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.478590 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.479733 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.472211 16854 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:57.482041 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.482455 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:57.468097 16860 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.480451 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.483363 16467 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:57.501648 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.507500 12062 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:57.480944 16859 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:57.512959 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.447646 16935 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.482980 16653 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:16:57.525735 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:57.539636 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 154738us
I20260709 13:16:57.540622 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:57.493659 16578 consensus_queue.cc:1048] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:16:57.509812 16467 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.509083 16854 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:57.513646 16092 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.545490 11334 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.521456 16793 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.526288 16815 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:57.520905 16850 consensus_queue.cc:1243] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:57.541926 16951 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:16:57.546531 16843 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:57.547376 16467 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:57.548951 16854 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:57.502224 16774 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.593997 11515 raft_consensus.cc:1240] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 10. Current term is 11. Ops: []
W20260709 13:16:57.599905 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.600534 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:57.600711 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.555475 16952 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.560192 16653 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.563534 16782 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.558233 16785 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:16:57.592622 16899 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.560294 16815 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.630338 16815 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.600066 16800 consensus_queue.cc:1059] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:16:57.646857 16815 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:57.594563 16859 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.673170 11441 raft_consensus.cc:1275] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:57.675930 12065 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:57.628682 16899 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.685261 16887 consensus_queue.cc:1048] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:57.678113 16898 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:57.686661 16838 consensus_queue.cc:1059] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:57.687141 16898 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.687932 16838 raft_consensus.cc:3055] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:57.688359 16838 raft_consensus.cc:740] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.689522 16898 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.695425 11439 raft_consensus.cc:1240] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:57.653723 16793 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.669845 16943 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14.
I20260709 13:16:57.695346 16920 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:57.724671 16920 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.727057 16920 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:57.732789 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 446955us
I20260709 13:16:57.733661 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.734702 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:16:57.712082 16956 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:57.740288 11368 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:16:57.738590 11518 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:57.739270 11440 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:57.763877 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:57.764534 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.765307 11409 leader_election.cc:400] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:57.766162 11409 leader_election.cc:403] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:57.705662 16883 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.770449 16898 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.739933 16578 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.786450 16578 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.789047 16578 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.790081 16898 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:57.776219 11362 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:16:57.689286 16838 consensus_queue.cc:260] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.778120 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.773665 16828 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.781351 11369 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:16:57.808820 16935 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:57.627965 16782 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.809520 16828 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.810400 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.783849 16899 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:57.739677 16630 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:16:57.816450 16828 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:57.817162 11409 leader_election.cc:400] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 16. Current term is 17.
I20260709 13:16:57.783540 16925 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.818783 11369 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:57.819334 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.824059 16925 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.819713 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.825129 11361 raft_consensus.cc:1813] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Handling vote request from an unknown peer 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.825568 11361 raft_consensus.cc:2368] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:57.823675 11409 leader_election.cc:403] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:57.827605 16630 consensus_queue.cc:1059] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:57.828537 16630 raft_consensus.cc:3055] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:57.828631 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:57.828919 16630 raft_consensus.cc:740] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.829926 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:57.829669 16630 consensus_queue.cc:260] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.831418 16920 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 16. Current term is 17.
I20260709 13:16:57.832114 16812 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:57.845254 16812 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.846112 16812 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.847121 16812 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:57.847798 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.850193 11369 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:57.850564 11369 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:57.851246 11369 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.853098 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.855268 16892 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:57.856060 16925 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.833909 16899 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:57.834518 16906 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:57.835134 16943 raft_consensus.cc:2764] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:16:57.835847 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:57.858389 11439 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:57.838809 12065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "116f1226682347599c239ad3bee95b41"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:57.859822 12065 raft_consensus.cc:606] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.860188 12065 raft_consensus.cc:614] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:57.860637 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:57.840306 11442 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:57.842213 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.843127 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:57.863484 16850 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:57.863924 16850 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.864238 16850 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:57.855960 16892 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:57.868326 16892 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:57.868714 16892 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.789965 16467 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.869587 16892 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.871143 16892 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:57.871578 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:16:57.871932 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.872409 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.872888 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.873231 12065 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:57.874321 11439 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:57.874733 16906 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:57.875420 16906 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:57.858655 16931 consensus_queue.cc:1059] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:16:57.877130 16811 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:16:57.877496 16811 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:57.844758 11519 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:57.811074 16782 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.833261 11483 leader_election.cc:400] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:57.879650 11483 leader_election.cc:403] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:57.839284 16630 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:57.871042 16467 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.884677 12066 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:57.868814 16832 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.888213 12061 raft_consensus.cc:1240] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:57.890007 16832 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:57.891096 11408 leader_election.cc:336] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Timed out: RequestConsensusVote RPC to 127.10.4.67:46667 timed out after 1.957s (SENT)
I20260709 13:16:57.891589 11408 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:57.895655 16966 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:16:57.899586 16907 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:16:57.895490 16803 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.901053 16844 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:57.874295 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:57.902065 16954 consensus_queue.cc:1175] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6e87b213a4554d94825aceec352fc094" responder_term: 14 status { last_received { term: 13 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:57.902817 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.905555 11361 raft_consensus.cc:2393] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:57.907752 16967 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:57.878306 16811 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.908521 16794 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:57.901933 16803 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:57.909086 16794 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:57.912290 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 79135.8us
I20260709 13:16:57.844906 11262 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:57.914891 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:57.919063 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:57.911181 16794 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.924628 11262 catalog_manager.cc:5697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:57.925462 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:57.926235 11367 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:16:57.930142 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:57.928931 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:16:57.924183 16756 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:16:57.937939 16665 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:57.938891 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:57.940034 16594 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:57.908681 16811 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:16:57.942812 11367 raft_consensus.cc:1275] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:16:57.943701 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.949106 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:57.922606 16846 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:16:57.944438 11262 catalog_manager.cc:5697] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 3 to 12, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:57.944185 16899 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.952055 16712 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:57.953594 16967 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:16:57.970816 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.951496 16850 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.974036 16850 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.968235 16712 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.974862 16712 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:57.952518 16933 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:16:57.957391 16943 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:57.987789 16943 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.993965 16707 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:57.994138 16860 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.994969 16707 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:57.995057 16712 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:57.996397 16860 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:58.007328 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 48226.9us
W20260709 13:16:58.013844 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 51716.3us
I20260709 13:16:57.997049 16712 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:57.971131 11262 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 15 to 16, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 16 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:57.997831 11361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "857937fa61dc400a81345944f0a14511"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:57.968160 16899 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:58.025591 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:58.027822 11440 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:57.970070 16967 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:16:58.017584 16978 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:16:58.034982 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 67870.6us
I20260709 13:16:58.042236 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.042371 11408 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:58.043474 16756 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:58.043886 16756 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:58.044706 16966 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:58.045111 16756 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:58.037062 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.050967 11439 raft_consensus.cc:1240] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 13:16:58.052482 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.032831 11262 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.057737 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:16:58.059352 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.064925 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:58.065848 11262 catalog_manager.cc:5697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:58.061319 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:58.073026 11365 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:16:58.077453 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:58.078224 12064 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 17: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:58.079687 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:58.080294 12066 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:58.080981 12067 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:16:58.081462 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:58.082080 12069 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:58.083674 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2629) took 2027 ms. Trace:
I20260709 13:16:58.086712 11440 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:58.089538 11361 raft_consensus.cc:606] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:58.062743 16975 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:16:58.089941 11361 raft_consensus.cc:614] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:58.092585 11262 catalog_manager.cc:5697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.087671 11335 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:58.008010 16854 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.084082 11511 rpcz_store.cc:276] 0709 13:16:56.055974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.170595 (+114621us) service_pool.cc:224] Handling call
0709 13:16:57.400315 (+1229720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.449190 (+ 48875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.083503 (+634313us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002a80d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fd8ad 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:58.083624 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2827547392}
W20260709 13:16:58.094520 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.071110 16832 consensus_queue.cc:1048] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:58.090227 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:58.034333 16967 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:58.091876 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:58.065512 16793 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.073995 16859 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:58.098743 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:58.031435 16937 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:16:58.134339 12069 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:58.077631 16712 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:58.085237 16630 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.078549 16876 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.116238 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.110515 16966 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:58.093104 16945 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:16:58.125257 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.155920 11440 raft_consensus.cc:1813] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Handling vote request from an unknown peer 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:58.129448 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:58.156513 11440 raft_consensus.cc:2368] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:58.094177 11262 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 7 to 8, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 8 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:16:58.131176 16803 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:58.108670 16661 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:58.098152 16774 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:58.090574 16594 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:16:58.126847 16943 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.118592 16860 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:16:58.159425 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.157635 11484 leader_election.cc:400] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
I20260709 13:16:58.165709 11366 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:58.173527 12034 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:58.098026 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:58.175920 11438 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:58.116094 16793 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:58.158211 16774 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:58.152509 16920 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:58.214919 16920 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.217204 16920 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.170046 16892 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:58.222368 16846 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 10. Current term is 11.
W20260709 13:16:58.229295 16982 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:16:58.229661 11333 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:58.231200 16786 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:58.231700 16786 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:58.232498 16786 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:58.184978 16933 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:16:58.236383 11512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36658 (request call id 2019) took 1754 ms (client timeout 1957 ms). Trace:
W20260709 13:16:58.236778 11512 rpcz_store.cc:269] 0709 13:16:56.481483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.483067 (+  1584us) service_pool.cc:224] Handling call
0709 13:16:58.236203 (+1753136us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54001504d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:16:58.236329 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3391782144}
I20260709 13:16:58.237237 16793 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:58.238960 16854 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.158321 16966 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.241976 11367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:58.242609 11367 raft_consensus.cc:606] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:58.242952 11367 raft_consensus.cc:614] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:58.243220 16966 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.243822 16756 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.246258 16756 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.157263 16876 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.263801 11439 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:16:58.154850 16630 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.158001 16803 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.246634 11367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:16:58.269968 11367 raft_consensus.cc:606] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:16:58.229939 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:58.246994 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 67151.8us
I20260709 13:16:58.256544 11262 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.267691 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:16:58.271481 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.271932 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:58.275203 11361 raft_consensus.cc:1240] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:58.285336 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.237502 16899 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:58.288652 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.289157 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:58.289397 11512 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:58.291103 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.293532 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:58.294369 12069 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:58.301571 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.302284 11438 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:58.304804 11439 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:16:58.310977 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.325771 16858 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:58.327219 11438 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:58.328013 11436 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:16:58.329149 16630 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.330891 16803 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.332484 16661 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.333835 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.343161 11439 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:58.306667 16650 raft_consensus.cc:3055] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:58.267966 16850 consensus_queue.cc:1059] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:16:58.264184 16336 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:16:58.298765 16846 consensus_queue.cc:1059] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:16:58.310480 16899 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.342095 16858 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.346069 16650 raft_consensus.cc:740] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:58.302130 11262 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.296521 16808 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:16:58.341965 16933 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:16:58.363948 11511 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:58.379531 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:58.380291 12069 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:58.368217 11437 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:58.383074 11334 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:58.384734 11262 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 10, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 10 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.387569 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2678) took 2172 ms. Trace:
I20260709 13:16:58.387974 11368 rpcz_store.cc:276] 0709 13:16:56.215120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.600613 (+385493us) service_pool.cc:224] Handling call
0709 13:16:56.600931 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:57.744759 (+1143828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.387301 (+642542us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002da854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:58.387513 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2619744640}
I20260709 13:16:58.391819 16800 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:58.392307 16800 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.392678 16800 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:58.429450 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:58.430107 12065 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:58.360841 16925 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:58.429276 11262 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.345875 16661 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:58.378147 16793 raft_consensus.cc:3055] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:58.440390 11262 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.354534 16630 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 13:16:58.431258 16971 log.cc:927] Time spent T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:16:58.449246 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.380391 16812 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:16:58.359748 16899 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.400633 16800 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:58.440438 16974 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:16:58.450666 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 130983us
I20260709 13:16:58.456413 11438 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:16:58.456149 11262 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 11, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.456358 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:58.457249 11511 raft_consensus.cc:1240] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:16:58.450935 16983 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:16:58.349273 16774 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.443208 16793 raft_consensus.cc:740] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:58.464501 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.468703 11512 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:58.468925 11517 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:16:58.469815 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:58.472009 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.472602 11512 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:58.467039 11262 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 12, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.473431 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.474020 11512 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
W20260709 13:16:58.466243 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 105503us
I20260709 13:16:58.466872 11484 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:16:58.475311 11512 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
W20260709 13:16:58.466904 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 83934.4us
I20260709 13:16:58.477654 11514 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:58.483783 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.486228 11510 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:16:58.482941 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.490556 11512 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:58.492391 11262 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 9, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 9 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.458487 16846 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:16:58.499104 11262 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.492200 16876 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:58.520445 11517 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:16:58.508940 11262 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.532953 16967 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.534233 11262 catalog_manager.cc:5697] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 11, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 11 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.536149 11262 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.504017 16966 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:58.537734 16966 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:58.537936 11262 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 11, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:16:58.538552 16966 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:58.521812 16793 consensus_queue.cc:260] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.540145 11262 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.540756 16793 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:58.519071 16803 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:16:58.502332 16860 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:58.527547 16661 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:58.529742 16707 consensus_meta.cc:287] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.532s	user 0.003s	sys 0.005s
I20260709 13:16:58.532316 16899 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:58.594239 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88455.3us
W20260709 13:16:58.601684 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.646222 11514 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:16:58.646649 11514 pending_rounds.cc:85] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 8
I20260709 13:16:58.646870 11514 pending_rounds.cc:107] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:16:58.647089 11514 raft_consensus.cc:2889] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:58.535087 16967 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.595237 16860 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.521486 16955 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
W20260709 13:16:58.657606 16985 log.cc:927] Time spent T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:16:58.658151 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:58.659389 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.659893 11333 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:16:58.660807 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:58.664081 17003 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:58.669263 17005 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:58.669768 17005 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:58.670105 17005 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:58.673403 16803 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:58.673856 16803 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.674180 16803 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:58.656193 16860 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:58.642411 16876 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.679728 16876 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.681881 16876 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:58.683503 16800 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.652415 16926 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.684490 16926 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:58.656505 16966 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:58.693166 16858 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.656942 16952 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.695757 16952 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.698354 16952 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:58.702072 16988 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.251s	user 0.011s	sys 0.000s
I20260709 13:16:58.703970 11515 raft_consensus.cc:1275] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:16:58.706805 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.707701 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:58.708455 11517 raft_consensus.cc:2393] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 17: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:16:58.709569 11510 raft_consensus.cc:1275] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:16:58.709925 11510 pending_rounds.cc:85] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 11
I20260709 13:16:58.710124 11510 pending_rounds.cc:107] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:16:58.710326 11510 raft_consensus.cc:2889] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:58.711546 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:58.712142 11518 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:58.720432 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.721393 11440 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:16:58.723645 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:58.725281 11444 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:58.729887 11335 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:58.732892 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:58.733888 12060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1341) took 1257 ms. Trace:
I20260709 13:16:58.734264 12060 rpcz_store.cc:276] 0709 13:16:57.476576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.479061 (+  2485us) service_pool.cc:224] Handling call
0709 13:16:57.479332 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.547084 (+ 67752us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:57.616021 (+ 68937us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.905730 (+289709us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.905790 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:57.905843 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.733681 (+827838us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b54005127d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:16:58.733726 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:58.733838 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24150,"prepare.queue_time_us":74525,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":34814,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":3578,"spinlock_wait_cycles":155293056,"thread_start_us":325940,"threads_started":2,"wal-append.queue_time_us":262620}
I20260709 13:16:58.743103 11440 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:16:58.756832 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:58.770200 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.773131 11444 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:58.765852 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:58.780341 11444 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:16:58.752733 16782 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:58.801137 16782 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.801668 16782 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:58.803936 16926 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:58.785676 16881 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:16:58.786083 16886 consensus_queue.cc:1048] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:58.781503 16665 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:58.776904 16876 consensus_queue.cc:1059] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
W20260709 13:16:58.825613 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.826829 16665 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.832782 11444 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:58.836261 16665 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:58.838027 17012 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:58.838639 17012 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:58.840405 16881 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:58.843814 16876 raft_consensus.cc:3055] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:58.844223 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:16:58.869685 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 209853us
W20260709 13:16:58.876274 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.845880 16881 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.844250 16876 raft_consensus.cc:740] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:16:58.868846 16926 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:58.889920 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 109603us
I20260709 13:16:58.830724 16803 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:58.898669 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:58.900075 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:58.925381 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:58.884075 16881 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:59.016323 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 120836us
I20260709 13:16:58.890793 16926 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:58.934985 17013 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:16:58.931378 17012 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:58.910712 16532 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:16:58.912693 17008 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.889317 16774 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:16:58.912869 17018 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:16:58.911016 16793 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:59.016073 11514 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:16:58.926779 16876 consensus_queue.cc:260] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.018754 11518 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.022178 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.027842 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.025153 16712 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.019528 17016 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.127s	user 0.000s	sys 0.006s
I20260709 13:16:59.027366 17007 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.025758 16952 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.038918 16926 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.017335 16955 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.015929 16885 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.030400 16892 consensus_queue.cc:1048] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:16:59.060645 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.061691 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.062901 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.083184 12069 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:59.064137 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.068225 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:59.084674 12067 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:16:59.050552 16665 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.071259 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:59.068964 16850 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.020114 16808 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:59.067726 16812 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:59.031387 17005 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.066380 16793 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:59.064967 16712 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.080060 17012 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.048031 17008 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.090169 16812 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.140905 12066 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:16:59.075486 16952 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.131006 16926 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.073352 17007 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.090066 16850 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.129973 16803 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.132721 16808 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:59.087261 16885 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.097447 16860 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.086669 16955 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.145460 16812 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:59.166482 16712 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:16:59.168058 17021 log.cc:927] Time spent T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:16:59.169734 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.170293 11437 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:16:59.171487 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:59.172863 16712 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:59.173306 16712 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.173712 16712 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:59.174048 17028 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:59.157053 16931 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.174820 16931 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.175674 16892 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.176501 17029 log.cc:927] Time spent T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:16:59.176285 16892 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.177040 16931 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.178422 16892 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.178764 17007 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.179044 16850 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:16:59.186628 17009 log.cc:927] Time spent T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.423s	user 0.000s	sys 0.002s
I20260709 13:16:59.181308 16892 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.194751 16892 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.198019 16812 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.199819 16876 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:59.200295 16812 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.204281 16892 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.205610 16712 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.206301 16967 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:59.206630 16650 consensus_queue.cc:260] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.207860 16650 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.208726 16712 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.212653 11369 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:59.212949 17020 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:16:59.218709 17004 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.747s	user 0.000s	sys 0.001s
W20260709 13:16:59.219597 17019 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:16:59.219899 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:16:59.221033 16786 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:59.221460 16786 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.221830 16786 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:59.224645 16883 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.225194 16883 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.138448 17012 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.229691 11516 raft_consensus.cc:1240] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:59.230751 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:59.230852 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.231459 11439 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:59.232031 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.253764 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.255538 16756 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:59.255963 16756 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.256330 16756 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:59.259421 11512 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:59.264967 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.229211 17012 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.259038 16806 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.274935 16806 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.275658 16881 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.277172 16806 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.277784 16806 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:59.278208 16806 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:59.279521 16782 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.280004 16860 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.227192 16883 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.163328 16808 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.281608 16782 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.172941 16955 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.282967 16808 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.259657 16943 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:59.286013 11367 raft_consensus.cc:1240] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:16:59.288909 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.289073 16808 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.288921 12069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:16:59.289633 16808 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.290269 12069 raft_consensus.cc:606] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:16:59.290587 12069 raft_consensus.cc:614] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:59.289119 16933 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.288933 16943 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.292219 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.291975 16943 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:16:59.292649 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.295416 16943 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.295913 16933 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.297845 16933 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.289227 16250 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:59.299727 11518 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:59.300292 16250 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.302347 16250 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.299633 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:59.275310 17008 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:16:59.304960 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 207550us
W20260709 13:16:59.307333 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.307502 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.308889 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.309063 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:59.309684 12067 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.310034 12069 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: []
I20260709 13:16:59.312003 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.313241 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.314661 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.315356 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:16:59.316071 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.316745 12065 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:16:59.314092 12064 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:59.175815 16966 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:59.320012 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:59.323532 11361 raft_consensus.cc:2393] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:16:59.321759 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.324049 11443 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 9.
I20260709 13:16:59.322187 11440 raft_consensus.cc:2393] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:16:59.322638 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.304301 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:59.330636 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.284675 16786 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.275586 16885 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.336697 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.331331 11364 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:16:59.338992 11368 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:59.339514 11439 raft_consensus.cc:2468] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:16:59.303277 16808 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.278236 17007 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.341482 17012 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.303704 16793 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.357792 16952 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.358773 16665 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.330144 16966 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.361246 16966 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.363441 16782 raft_consensus.cc:993] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:59.363926 16782 raft_consensus.cc:1081] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:16:59.156787 16926 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.366528 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:59.367103 11513 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:59.367702 16926 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.281642 16892 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.352492 16885 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.368440 16892 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.370575 16892 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.371009 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.371652 11514 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:59.372776 17007 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.373283 16931 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:59.373869 16931 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.279109 16806 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:16:59.378031 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:59.378516 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.382750 17018 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:59.364012 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.383611 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:59.384202 11510 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:59.384553 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.337357 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.368494 16885 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.333591 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:59.386762 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.387428 11438 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:59.388298 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:59.336874 16945 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:59.260512 16967 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.384198 11444 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.385514 11366 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:16:59.356271 17012 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.394193 12067 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:59.392053 11513 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
W20260709 13:16:59.402191 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 59566.8us
I20260709 13:16:59.403286 11333 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:59.409004 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 65201.8us
I20260709 13:16:59.418692 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.419461 11366 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:16:59.272910 16756 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.437871 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.446130 11444 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:59.394232 16966 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:59.429981 16532 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.440729 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:59.432686 16803 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:59.450469 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.450465 11369 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:16:59.451763 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.453528 11440 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:59.429773 16943 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:59.456853 11334 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.457917 12064 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:16:59.464076 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.458037 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.463982 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.465173 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.465782 12065 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:59.468039 11334 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.468427 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:59.469539 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.470436 11483 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.471180 15852 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:59.399817 17018 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.462389 11369 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1351) took 1616 ms. Trace:
I20260709 13:16:59.471920 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.426760 16952 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:16:59.406491 16945 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.478813 16756 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.479779 16945 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.480337 16945 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:59.481004 16945 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:59.481344 16945 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.481644 16945 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:59.475852 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:59.485457 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.486467 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.486974 11440 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:59.487864 16885 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.433771 16782 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:16:59.489609 16782 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:59.489944 16782 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:59.490679 16782 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.493167 17018 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.495426 16838 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:59.498477 16945 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.475924 11369 rpcz_store.cc:276] 0709 13:16:57.845922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.849811 (+  3889us) service_pool.cc:224] Handling call
0709 13:16:57.850086 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.212204 (+1362118us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b58008636d0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc95a0aa 00007f35bc9dc447 00007f35bc9f4fe7 00007f35bc9f67de 00007f35bca00b49 00007f35bca01416 00007f35bc9fdd8f 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4
0709 13:16:59.461863 (+249659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.462332 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003000576}
I20260709 13:16:59.508081 16945 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.509859 16843 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:59.510320 16843 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:59.463119 16532 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.516871 16532 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.518146 16786 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.519590 17012 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.521023 16967 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.467162 16812 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:16:59.522668 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.523371 12061 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:59.526602 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.527252 12064 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:59.527328 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.528817 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.530268 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.525986 11361 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:16:59.534721 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.533058 12034 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.462132 17006 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.536504 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.537070 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.539919 11510 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:16:59.540324 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.541213 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.541873 12067 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:16:59.475713 16785 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:59.522018 16812 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.549015 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.476369 16931 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.550401 16578 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:59.456077 16966 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.536177 16933 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.537982 17012 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:59.540083 16925 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:59.575168 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.575161 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.467342 16943 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.576980 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.536093 16858 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:16:59.543188 17005 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.578516 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:16:59.523826 16844 consensus_queue.cc:1059] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:16:59.552148 16785 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.570209 16812 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:59.622785 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 50068.1us
I20260709 13:16:59.606781 11438 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:59.626854 11408 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.571215 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.511123 16843 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.556263 16709 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:59.572847 16578 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.585445 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.678484 16578 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.606267 11443 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
W20260709 13:16:59.612834 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 79828.6us
I20260709 13:16:59.577478 16782 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:59.641921 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 70795.3us
I20260709 13:16:59.689327 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.625154 12033 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.655818 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:59.691215 11513 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.691821 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.692610 11511 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:16:59.693665 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.694312 11511 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:59.695822 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.696169 17005 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.696817 11519 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:59.696676 17005 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.698694 17005 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:16:59.701493 16250 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:59.701985 16250 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.702338 16250 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.657675 11408 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.622582 16844 consensus_queue.cc:1243] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:59.547513 17006 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.663893 16709 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.707731 16709 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:59.707962 16510 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:16:59.580791 17012 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.712383 17012 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.677943 16843 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.659765 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.728029 11515 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:59.695132 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:16:59.739713 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 82099.3us
I20260709 13:16:59.747923 11515 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
W20260709 13:16:59.750864 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 61995.8us
I20260709 13:16:59.751293 15852 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.751802 15852 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.622947 16785 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.622248 16811 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:59.650105 16786 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:59.665879 16925 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:59.762228 11511 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:16:59.650768 16756 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:16:59.601644 16933 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.768281 11513 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:16:59.741891 11517 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:36660
I20260709 13:16:59.747668 16967 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:59.739660 16797 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.778875 11517 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:16:59.765110 17024 log.cc:927] Time spent T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:16:59.782346 11517 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:16:59.778954 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 338237us
I20260709 13:16:59.785796 15852 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.757792 16811 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:16:59.795554 11438 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:59.796134 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.796886 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.797446 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.798061 11369 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:16:59.799031 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.773373 17007 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.776432 16925 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.833596 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:59.799659 11370 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:59.834309 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.835281 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.803907 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.804229 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.811105 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:16:59.811091 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:16:59.839352 11438 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:16:59.839473 12066 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
W20260709 13:16:59.818850 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:59.809542 11517 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.840370 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.841457 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.787441 16844 raft_consensus.cc:493] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.794188 17041 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:59.846529 11517 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:16:59.802033 16811 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:59.777330 16786 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.852749 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.837255 16925 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.851226 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:59.856793 11511 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:59.856875 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.833089 16712 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.850996 11484 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:59.853958 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:59.854718 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.841012 11366 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:59.786731 16756 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.857473 11367 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
W20260709 13:16:59.854110 11409 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.016s	sys 0.012s
I20260709 13:16:59.788866 16797 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.863991 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:16:59.849215 16742 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.862526 11364 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:16:59.867861 11335 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:16:59.865906 11409 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.017s	sys 0.013s
W20260709 13:16:59.869239 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 615503us
I20260709 13:16:59.870038 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:16:59.870692 11367 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:16:59.870983 11367 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:59.866811 11363 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:16:59.874190 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:16:59.874056 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.869107 11335 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:59.865113 16712 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.886762 16712 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:16:59.848917 16967 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:16:59.854656 16881 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:16:59.871940 11367 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.893293 11335 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:59.897097 11511 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 18.
W20260709 13:16:59.860634 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 392633us
I20260709 13:16:59.860369 17041 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:16:59.890928 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:16:59.905519 11367 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:16:59.906157 11334 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:16:59.910854 12069 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:59.872828 16858 raft_consensus.cc:2804] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:59.913483 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:16:59.850447 16844 raft_consensus.cc:515] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.881949 16854 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:59.962580 11511 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
W20260709 13:16:59.961326 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68100.9us
I20260709 13:16:59.882675 16899 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:59.961530 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 55493.8us
W20260709 13:16:59.970783 17035 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:16:59.966408 17048 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:59.971360 17048 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.971778 17048 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.967914 11370 raft_consensus.cc:2410] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:16:59.973099 11436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2903) took 1247 ms. Trace:
I20260709 13:16:59.968283 16794 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:59.973711 16794 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.970871 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.968493 11333 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:16:59.968894 16858 raft_consensus.cc:697] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:16:59.908227 17005 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:16:59.889258 16742 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.990104 16858 consensus_queue.cc:237] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:16:59.990976 11362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47990 (request call id 1356) took 1984 ms. Trace:
W20260709 13:16:59.991894 17034 log.cc:927] Time spent T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:16:59.904597 17007 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.988466 17041 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:16:59.973497 11436 rpcz_store.cc:276] 0709 13:16:58.726235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.095491 (+369256us) service_pool.cc:224] Handling call
0709 13:16:59.095794 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.096140 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.096162 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.119360 (+ 23198us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.542760 (+423400us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:59.542803 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.542873 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.972918 (+430045us) raft_consensus.cc:1685] finished
0709 13:16:59.972937 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.973050 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":352141,"threads_started":1,"wal-append.queue_time_us":620561}
I20260709 13:16:59.974226 16794 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:59.888572 16803 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:16:59.908095 16793 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:17:00.012064 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 43639.5us
I20260709 13:17:00.025127 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:16:59.911783 16966 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.013707 11362 rpcz_store.cc:276] 0709 13:16:58.006768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.007059 (+   291us) service_pool.cc:224] Handling call
0709 13:16:58.007343 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:58.027197 (+ 19854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:58.027228 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:58.389193 (+361965us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b1800162760. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a
0709 13:16:59.228516 (+839323us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b2c000c7fc8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:16:59.228994 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.720858 (+491864us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:59.720894 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.720959 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.990750 (+269791us) raft_consensus.cc:1685] finished
0709 13:16:59.990772 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.990903 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10276,"raft.queue_time_us":19915,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5476,"spinlock_wait_cycles":9901312,"thread_start_us":491451,"threads_started":1,"wal-append.queue_time_us":653018}
I20260709 13:16:59.919631 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:00.027357 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:00.027642 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 108102us
I20260709 13:16:59.961139 16712 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:59.962798 16886 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.011s
I20260709 13:17:00.029716 11484 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:16:59.912129 16881 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.018011 11367 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:17:00.014344 16943 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.084156 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54541.6us
I20260709 13:17:00.010414 16782 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:00.132417 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.029302 16806 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:00.066437 16838 consensus_meta.cc:287] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.002s
I20260709 13:17:00.130257 11511 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 13. Current term is 15. Ops: []
I20260709 13:17:00.139295 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.133160 11436 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:17:00.023837 16793 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.029704 16712 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.135272 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.147025 11438 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:00.135890 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 510797us
W20260709 13:17:00.136322 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.011633 16854 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:00.068989 16742 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.143653 11484 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:00.020664 16954 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:59.909912 16967 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.150440 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.130893 16887 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:00.167857 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.062341 16926 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:00.085618 17050 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.065399 16250 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.140969 16874 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:00.085000 16886 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.009s	sys 0.012s
I20260709 13:17:00.159548 16712 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.200619 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:00.260475 12063 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:00.220744 11438 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
I20260709 13:17:00.198627 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.078578 16774 consensus_queue.cc:1059] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 13:17:00.068032 16933 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:00.063220 17012 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.255741 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.262256 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:00.264333 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 64683.3us
I20260709 13:17:00.151103 16793 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.026919 17007 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.278918 17007 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.063920 16803 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.279830 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.276037 16756 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.194867 16954 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:00.268702 12062 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
W20260709 13:17:00.270082 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 112322us
W20260709 13:17:00.270355 17045 log.cc:927] Time spent T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:17:00.219872 16926 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.184587 17008 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:00.192291 16846 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:00.275032 16943 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:00.276033 17044 log.cc:927] Time spent T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 13:17:00.282850 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:00.186215 16837 consensus_queue.cc:1059] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
W20260709 13:17:00.253907 16709 consensus_meta.cc:287] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.007s
I20260709 13:17:00.302011 11515 raft_consensus.cc:2368] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:17:00.273841 11442 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:17:00.149376 16806 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.259868 16937 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.283313 16936 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:00.330245 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 47472.8us
I20260709 13:17:00.355623 12033 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:00.294942 16926 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:00.293380 16846 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.301048 17052 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.282092 16954 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.323050 16887 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:00.264568 17050 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.261129 16874 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.370980 16846 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:00.368335 16837 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:00.298360 17008 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.392306 16793 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.396019 12068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1363) took 1082 ms. Trace:
I20260709 13:17:00.301290 16943 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.409365 12068 rpcz_store.cc:276] 0709 13:16:59.313822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.313960 (+   138us) service_pool.cc:224] Handling call
0709 13:16:59.314263 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.320360 (+  6097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.320405 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.320900 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.756341 (+435441us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:59.756380 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.756443 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:00.395784 (+639341us) raft_consensus.cc:1685] finished
0709 13:17:00.395804 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.395961 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5576,"raft.queue_time_us":6331,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":7107,"thread_start_us":382704,"threads_started":1,"wal-append.queue_time_us":561375}
W20260709 13:17:00.397516 16925 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.532s	user 0.007s	sys 0.001s
W20260709 13:17:00.391456 16812 consensus_meta.cc:287] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.000s
I20260709 13:17:00.389236 17040 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:00.416006 16887 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:00.441931 16812 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.442466 17050 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.405155 17052 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.420845 16933 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.493666 16933 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.495774 16933 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.499254 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.441995 16846 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.499588 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.499764 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.500139 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.500305 11363 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
W20260709 13:17:00.445262 17059 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:17:00.448757 17006 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.452570 16806 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.454515 17033 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.535s	user 0.000s	sys 0.001s
I20260709 13:17:00.456012 16250 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.456526 16844 leader_election.cc:290] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.457458 17056 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
I20260709 13:17:00.461599 12069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1358) took 1150 ms. Trace:
I20260709 13:17:00.464875 16874 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.469879 11409 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.4.68:37891 timed out after 1.881s (SENT) [suppressed 15 similar messages]
I20260709 13:17:00.471956 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:00.474802 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:17:00.532225 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 57506.7us
I20260709 13:17:00.532752 11510 raft_consensus.cc:2368] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:17:00.476359 17012 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.395895 16837 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.329281 16709 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.533191 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:00.485769 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 223587us
I20260709 13:17:00.533476 17012 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:00.533864 17012 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:00.534214 17012 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:00.534861 16709 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.399194 17008 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.535509 16837 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:00.536958 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.487672 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:00.487736 16887 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.482715 16925 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.541646 16925 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.430575 16843 consensus_meta.cc:287] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.677s	user 0.007s	sys 0.003s
I20260709 13:17:00.362417 16854 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.543205 11512 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:00.357298 16937 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.359983 16936 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.491925 16967 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.421557 17023 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:00.395495 17063 raft_consensus.cc:3055] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:00.396401 16881 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:00.497838 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 472762us
W20260709 13:17:00.437822 17043 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
W20260709 13:17:00.444428 17047 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:17:00.501628 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.502815 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.503629 11366 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:00.504650 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.604943 11333 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:00.605305 11367 raft_consensus.cc:2410] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:17:00.497126 16578 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.818s	user 0.011s	sys 0.000s
W20260709 13:17:00.508481 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 232165us
I20260709 13:17:00.528228 12069 rpcz_store.cc:276] 0709 13:16:59.311332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.311504 (+   172us) service_pool.cc:224] Handling call
0709 13:16:59.311800 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.911299 (+599499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.461334 (+550035us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b5400656b54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:00.461533 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377284864}
I20260709 13:17:00.586092 17052 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.539740 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.553565 16712 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.617161 12069 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:00.617444 16578 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.618562 16712 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:00.618695 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.619153 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:00.619925 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.620177 12068 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:00.620536 12069 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:17:00.620837 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.621047 16594 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:00.620527 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.621061 11513 raft_consensus.cc:2368] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:17:00.621585 16594 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.612977 16250 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:00.622694 16594 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:00.623045 16250 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.621564 17032 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.624984 17032 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.625265 16250 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.625967 17084 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:00.554369 17046 log.cc:927] Time spent T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:17:00.556685 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.558338 16837 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.560360 11437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2906) took 1704 ms. Trace:
I20260709 13:17:00.573339 16943 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.574865 16846 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.576143 16785 consensus_meta.cc:287] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.802s	user 0.010s	sys 0.000s
W20260709 13:17:00.525966 17048 consensus_meta.cc:287] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.004s
I20260709 13:17:00.576887 16812 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:00.574816 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.619454 16578 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.525431 17006 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:00.496369 17040 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.529165 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.610163 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:00.613312 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.645601 12060 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:17:00.648846 16837 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.614229 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.651050 17006 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:00.615550 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 82423.5us
I20260709 13:17:00.628214 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.629629 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.629297 11367 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:17:00.597478 17023 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:00.632102 11437 rpcz_store.cc:276] 0709 13:16:58.856449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.171007 (+314558us) service_pool.cc:224] Handling call
0709 13:16:59.171308 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.171962 (+   654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.171991 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.172471 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.407487 (+235016us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:59.407528 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.407589 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:00.453881 (+1046292us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:17:00.560054 (+106173us) raft_consensus.cc:1685] finished
0709 13:17:00.560076 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.560298 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":4648192,"thread_start_us":234603,"threads_started":1,"wal-append.queue_time_us":670779}
I20260709 13:17:00.633222 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.596776 17063 raft_consensus.cc:740] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:00.635051 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.621647 16844 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:00.602496 16793 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.639621 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.641433 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:00.632417 16794 consensus_meta.cc:287] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.000s
I20260709 13:17:00.651965 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.636319 17048 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.650487 16594 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.653115 17006 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.653420 11363 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:00.661187 16937 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:00.662492 11518 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1234) took 1311 ms. Trace:
I20260709 13:17:00.653932 11362 raft_consensus.cc:2410] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:17:00.662714 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.654886 11514 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:00.676155 11443 raft_consensus.cc:2410] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:17:00.667887 16936 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.675273 11518 rpcz_store.cc:276] 0709 13:16:59.350621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.351350 (+   729us) service_pool.cc:224] Handling call
0709 13:16:59.426182 (+ 74832us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.434041 (+  7859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.434075 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.434551 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.875238 (+440687us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:59.875282 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.875360 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:00.662244 (+786884us) raft_consensus.cc:1685] finished
0709 13:17:00.662265 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.662431 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7561,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":440395,"threads_started":1,"wal-append.queue_time_us":987958}
I20260709 13:17:00.668556 11516 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48448 (request call id 1233) took 1319 ms. Trace:
I20260709 13:17:00.640877 16785 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.678479 12066 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:00.679338 16785 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.679939 16594 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:00.680160 11369 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:00.677551 11516 rpcz_store.cc:276] 0709 13:16:59.348794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.349957 (+  1163us) service_pool.cc:224] Handling call
0709 13:16:59.350247 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.440625 (+ 90378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.440653 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.449948 (+  9295us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.864394 (+414446us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:59.864437 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.663198 (+798761us) spinlock_profiling.cc:243] Waited 90 ms on lock 0x7b540014b9d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:00.663250 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:00.663325 (+    75us) raft_consensus.cc:1685] finished
0709 13:17:00.663340 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.668489 (+  5149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":455,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":9096,"spinlock_wait_cycles":207096192,"thread_start_us":380960,"threads_started":1,"wal-append.queue_time_us":1007379}
I20260709 13:17:00.654173 17023 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:00.655619 17063 consensus_queue.cc:260] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.673928 11438 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.688966 17063 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:00.582115 12063 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 18.
I20260709 13:17:00.656826 11436 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:00.675185 17048 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:00.634012 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 164163us
I20260709 13:17:00.677095 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.696280 11443 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:17:00.657351 11519 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 17: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:00.697280 16943 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.657792 16794 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.697774 16943 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.698750 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.695924 17080 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.699059 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:00.655982 11444 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:17:00.700910 17080 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.701447 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:00.704504 11513 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:00.700181 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.659262 11440 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:17:00.706004 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:00.653689 16811 consensus_meta.cc:287] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: Time spent flushing consensus metadata: real 0.790s	user 0.007s	sys 0.001s
I20260709 13:17:00.653649 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:00.706934 12034 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:00.708593 17041 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:00.708673 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.711817 11443 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:17:00.711761 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 58176.6us
I20260709 13:17:00.650427 17012 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.710026 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:00.713455 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:00.723080 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.729224 11362 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.723444 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.734792 11438 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:00.728001 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.728209 12064 raft_consensus.cc:2393] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 17: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:00.744184 11363 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:00.722415 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.745242 11369 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.727695 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.748034 11367 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:00.744123 11361 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.724896 11436 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:17:00.700191 16943 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.737416 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 127440us
I20260709 13:17:00.760769 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:00.742871 11444 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:17:00.761602 12063 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 16: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:00.762400 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:00.763248 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.762835 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:00.758639 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:00.766340 11366 raft_consensus.cc:3055] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:00.759992 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.766727 11366 raft_consensus.cc:740] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:00.767326 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.701421 16794 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.768493 11367 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:00.728070 16846 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:00.658776 17040 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.769260 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.712343 17041 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:00.775105 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.775800 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:00.739471 16250 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:00.742767 16843 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:00.737165 16967 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:00.743129 16844 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:00.781519 11369 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:00.779579 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.777086 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:00.767452 11366 consensus_queue.cc:260] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.792384 11513 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:00.792809 11366 raft_consensus.cc:3060] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:00.739802 16966 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:00.812858 16966 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.814569 11445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2765) took 1019 ms. Trace:
I20260709 13:17:00.814860 16966 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.786298 11362 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:00.790729 11443 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:00.795644 16785 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:00.772922 16943 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:00.820705 16943 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.820806 11333 leader_election.cc:400] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:17:00.821090 16943 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:00.770855 16936 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:00.781085 16794 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:17:00.823161 11445 rpcz_store.cc:276] 0709 13:16:59.794851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.797810 (+  2959us) service_pool.cc:224] Handling call
0709 13:16:59.798066 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.891647 (+ 93581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.891678 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.891718 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:59.891756 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.814329 (+922573us) spinlock_profiling.cc:243] Waited 93.4 ms on lock 0x7b5400208fd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:00.814377 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.814507 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214712192}
I20260709 13:17:00.737142 17081 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:00.825301 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:00.797863 16926 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.827206 17012 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.827555 11511 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:00.801115 11366 raft_consensus.cc:1275] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:17:00.829568 11366 pending_rounds.cc:85] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 9
I20260709 13:17:00.829851 11366 pending_rounds.cc:107] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:17:00.796741 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:00.803383 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:00.769294 17006 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:00.713943 16811 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.821134 11333 leader_election.cc:403] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:00.830068 11366 raft_consensus.cc:2889] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:00.836143 11513 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:00.781873 17041 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.825555 17080 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:00.832873 11361 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:00.782035 16250 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.767849 16756 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:17:00.841223 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.827231 16926 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:00.823714 16936 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:00.843075 11361 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:00.834136 17040 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:00.835739 16594 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:00.829056 16844 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.847676 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.848307 11361 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:00.841874 16250 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.855340 11361 raft_consensus.cc:2468] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:17:00.852463 17048 raft_consensus.cc:3055] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:00.861032 17048 raft_consensus.cc:740] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:00.861903 17048 consensus_queue.cc:260] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.863125 17048 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:00.864547 16811 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.865135 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.865535 16936 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.866218 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.866854 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.904083 11438 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:00.869938 11484 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:00.871423 16951 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:17:00.847733 17040 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.841938 17052 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:17:00.852842 16594 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:00.873687 17023 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.877149 16797 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:00.901826 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 105130us
I20260709 13:17:00.917716 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:00.902455 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 81790.7us
I20260709 13:17:00.901858 11443 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:17:00.919545 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:00.911674 11438 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:00.905169 11484 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:00.922240 17023 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:00.924120 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.921557 16742 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:00.925736 16797 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:00.926123 16797 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.926163 16742 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:00.926604 16797 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:00.927508 16786 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.928606 12066 raft_consensus.cc:1240] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:17:00.929071 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:00.927166 16742 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:00.929729 12069 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:17:00.913118 16952 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:00.930523 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:00.873283 16712 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:17:00.932811 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.914418 16854 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:17:00.936632 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.937747 11408 leader_election.cc:400] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:17:00.937693 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad5a541019548a3b543619a92b420b1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.914031 16926 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:00.940804 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:00.941640 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:00.946382 11361 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 8.
I20260709 13:17:00.947609 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:00.924953 16954 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.002s	sys 0.012s
I20260709 13:17:00.947566 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:00.942323 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 105732us
I20260709 13:17:00.941983 11484 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:00.944643 11408 leader_election.cc:403] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:17:00.945500 11440 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:00.948411 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.951037 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:00.952028 11408 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:00.952257 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:00.917356 17055 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:00.955008 11511 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:00.956884 11444 raft_consensus.cc:1240] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:17:00.958549 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.959151 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:00.940044 16952 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.946475 16756 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:00.959889 11517 raft_consensus.cc:2393] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:00.963951 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:00.967358 11444 raft_consensus.cc:2393] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:00.952030 16794 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:00.953581 16806 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:17:00.953606 16774 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:00.980032 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:00.967343 11483 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:00.955832 16967 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:00.963881 17012 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:00.948619 16954 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.002s	sys 0.012s
I20260709 13:17:00.959856 16578 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:00.956730 17006 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:00.983086 16843 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:00.955950 16250 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.042462 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2711) took 1244 ms. Trace:
I20260709 13:17:01.036386 16844 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:00.965620 17081 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:17:01.043747 17041 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.069083 17041 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:01.037752 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.986990 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:01.069923 11443 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:17:01.070526 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:00.960031 16951 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:01.072809 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:00.965821 16854 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.049026 11517 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:17:01.077291 11443 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:00.956678 16846 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.036962 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.053665 11444 raft_consensus.cc:2393] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:01.078033 11440 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:17:00.981882 16794 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:00.983676 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:01.080317 11445 raft_consensus.cc:2393] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:17:01.036521 17065 log.cc:927] Time spent T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:17:00.980854 16650 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.056157 16578 raft_consensus.cc:493] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:00.963855 17093 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:01.110730 16578 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:00.955847 16926 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:17:01.054072 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 86786.9us
I20260709 13:17:01.111932 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:01.113157 17080 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:01.113545 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:01.113710 17080 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:01.114770 11483 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:01.064213 11368 rpcz_store.cc:276] 0709 13:16:59.797681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.797808 (+   127us) service_pool.cc:224] Handling call
0709 13:16:59.831437 (+ 33629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.831725 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.831756 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:00.828645 (+996889us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b2c000c7fc8. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bb2f7a11 00007f35bb307372 00007f35bca39a01 00007f35bc9fe43a 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2
0709 13:17:00.829061 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 13:17:00.914727 (+ 85666us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:00.914783 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.916311 (+  1528us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:01.042186 (+125875us) raft_consensus.cc:1685] finished
0709 13:17:01.042205 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.042405 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3537664,"thread_start_us":84620,"threads_started":1,"wal-append.queue_time_us":119569}
I20260709 13:17:01.066964 16844 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:00.964812 16952 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:01.062716 17094 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.086280 16650 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:01.093508 17037 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 1.233s	user 0.001s	sys 0.002s
I20260709 13:17:01.080565 16843 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.083487 16794 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:01.105086 16250 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:00.979127 16594 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.061283 16967 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:01.060881 16712 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:01.110812 17093 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.045351 16892 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:01.083964 16854 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.126902 11365 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:17:01.148516 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:01.124342 16874 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:01.124186 17012 raft_consensus.cc:2804] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:17:01.153563 11515 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2678) took 1279 ms (client timeout 1534 ms). Trace:
I20260709 13:17:01.154728 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.122038 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:01.195112 11445 raft_consensus.cc:3060] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:01.085580 16846 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.261232 11368 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
W20260709 13:17:01.224267 11515 rpcz_store.cc:269] 0709 13:16:59.873779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.873917 (+   138us) service_pool.cc:224] Handling call
0709 13:17:01.153383 (+1279466us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b540031f454. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:17:01.153510 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982187904}
I20260709 13:17:01.143170 16885 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.155752 17093 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:01.320633 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 153682us
I20260709 13:17:01.128474 17083 consensus_queue.cc:1048] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.134s
I20260709 13:17:01.261850 12061 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:17:01.327018 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1369) took 1128 ms. Trace:
W20260709 13:17:01.317463 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.317452 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 2905) took 2509 ms. Trace:
I20260709 13:17:01.114702 17080 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:01.309759 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 199917us
I20260709 13:17:01.338135 11445 raft_consensus.cc:2468] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:01.341257 16843 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.324831 16937 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:01.361510 16937 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.353376 11367 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:01.354375 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37086 (request call id 2719) took 1501 ms. Trace:
I20260709 13:17:01.324923 16885 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.362859 11370 rpcz_store.cc:276] 0709 13:16:59.853483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.853708 (+   225us) service_pool.cc:224] Handling call
0709 13:17:01.354152 (+1500444us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002dbc54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:17:01.354306 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244080896}
I20260709 13:17:01.325881 17040 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:01.299743 17100 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:17:01.315598 16874 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:01.366544 16874 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.369201 12034 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:01.371656 16931 raft_consensus.cc:2804] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:01.372208 16931 raft_consensus.cc:697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:01.373078 16931 consensus_queue.cc:237] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.377101 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.378268 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2754) took 1999 ms. Trace:
I20260709 13:17:01.378685 11439 rpcz_store.cc:276] 0709 13:16:59.378469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.379604 (+  1135us) service_pool.cc:224] Handling call
0709 13:16:59.379876 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.437487 (+ 57611us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:59.437741 (+   254us) log.cc:844] Serialized 14 byte log entry
0709 13:17:00.430775 (+993034us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:00.430833 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:00.430885 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.378024 (+947139us) spinlock_profiling.cc:243] Waited 57.4 ms on lock 0x7b5400398854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:01.378073 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.378208 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":529,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":1577,"raft.queue_time_us":89527,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":38373,"spinlock_wait_cycles":131959680,"thread_start_us":971393,"threads_started":2,"wal-append.queue_time_us":758582}
I20260709 13:17:01.130542 17094 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.167181 16712 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:01.220237 17081 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.330134 11441 rpcz_store.cc:276] 0709 13:16:58.807682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.096472 (+288790us) service_pool.cc:224] Handling call
0709 13:16:59.096754 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.135229 (+ 38475us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:59.135263 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.143702 (+  8439us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.546442 (+402740us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:59.546481 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.546548 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:01.092181 (+1545633us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:17:01.296982 (+204801us) raft_consensus.cc:1685] finished
0709 13:17:01.297006 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.317373 (+ 20367us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23369,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6727,"spinlock_wait_cycles":6627072,"thread_start_us":384680,"threads_started":1,"wal-append.queue_time_us":639054}
I20260709 13:17:01.169250 16892 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.327415 12061 rpcz_store.cc:276] 0709 13:17:00.198680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.198782 (+   102us) service_pool.cc:224] Handling call
0709 13:17:01.154605 (+955823us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9eee 0000000000375939 00007f35c194d1e0 00007f35c1a160c4 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:17:01.326837 (+172232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3574144}
I20260709 13:17:01.390232 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:01.319847 17070 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.171s	user 0.000s	sys 0.003s
I20260709 13:17:01.389999 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.389667 16899 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:01.399473 16650 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.152022 16967 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:01.150956 16943 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.420742 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.421342 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:01.421729 11510 raft_consensus.cc:2410] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:17:01.422290 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.322407 17012 raft_consensus.cc:697] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:01.419909 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:01.402874 11517 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:17:01.424620 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:01.420907 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.426561 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:01.428784 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.429527 11441 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:01.432439 11518 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:17:01.438176 11436 raft_consensus.cc:2368] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:17:01.420845 16886 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.437639 16250 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.445207 16250 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.435357 16832 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.447711 16832 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.448621 16943 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.449620 16832 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.399483 17081 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.454823 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.455577 11515 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:01.457437 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:01.451660 16797 consensus_meta.cc:287] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.003s
I20260709 13:17:01.439317 16885 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.439911 16936 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:01.423180 16899 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:01.394119 16926 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.465019 11439 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 11.
W20260709 13:17:01.471839 17036 log.cc:927] Time spent T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: Append to log took a long time: real 1.553s	user 0.006s	sys 0.002s
I20260709 13:17:01.474489 16899 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:01.453999 16832 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:01.478745 16967 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.489924 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.495847 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.453811 16886 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.464946 16797 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:01.520931 17104 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:17:01.491043 16832 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.528717 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:01.494285 16854 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.532946 16578 raft_consensus.cc:515] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.532998 16832 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.536010 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:01.536826 11441 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:01.537822 17039 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.538420 17039 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.540776 17039 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.543032 16578 leader_election.cc:290] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.547917 11516 raft_consensus.cc:2393] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:01.466638 16936 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:01.552276 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.512976 16250 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.556990 11366 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
W20260709 13:17:01.554670 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.555507 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.556272 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.491355 16952 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.559814 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.530325 16846 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.558027 11441 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:01.466570 16712 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.493971 17012 consensus_queue.cc:237] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.557349 16937 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.570540 11364 raft_consensus.cc:2368] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:01.571888 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.583346 12065 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
W20260709 13:17:01.584113 12065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1370) took 1385 ms (client timeout 1534 ms). Trace:
W20260709 13:17:01.584584 12065 rpcz_store.cc:269] 0709 13:17:00.198923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.199075 (+   152us) service_pool.cc:224] Handling call
0709 13:17:01.571759 (+1372684us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b5800037598. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a160ab 00007f35c1a16131 00007f35c19c40a0 00007f35c19c3984 00007f35baf3e6d4 00007f35baf3e662
0709 13:17:01.584042 (+ 12283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3712384}
I20260709 13:17:01.586256 12061 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:01.587354 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.588230 12063 raft_consensus.cc:2410] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:17:01.589305 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.590185 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.591955 12061 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:17:01.592988 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.593310 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.593452 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.594002 12065 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:01.594820 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.595464 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.595960 12061 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:01.595364 12064 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:01.585011 16756 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.616s	user 0.008s	sys 0.002s
I20260709 13:17:01.598352 12063 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 16: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:01.601294 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:01.603053 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.603703 12067 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:01.603802 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.602582 16756 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:01.621459 12065 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:01.637049 12067 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:01.617450 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:01.633320 17081 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.647074 17094 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.645361 17093 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.687860 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:01.689466 11441 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:01.701282 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.715435 11441 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:01.643945 16594 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:01.736703 16594 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:01.645547 16794 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.638749 16797 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.646227 16886 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.649449 16936 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.749954 17093 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.750766 17008 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:01.751358 17008 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:01.762274 16874 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.763216 11445 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:01.765777 17032 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:01.738967 16594 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.774570 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.775576 11511 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:17:01.776367 17032 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:01.777449 17032 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.783916 11484 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:01.787300 11442 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1236) took 1176 ms. Trace:
I20260709 13:17:01.787899 11442 rpcz_store.cc:276] 0709 13:17:00.611247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.614107 (+  2860us) service_pool.cc:224] Handling call
0709 13:17:00.614440 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:00.663378 (+ 48938us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:17:00.663775 (+   397us) log.cc:844] Serialized 14 byte log entry
0709 13:17:00.834798 (+171023us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:00.834843 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:17:00.834886 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.786774 (+951888us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b540027a6d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:01.786857 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.787094 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2092,"prepare.queue_time_us":199,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":13223,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":44780,"spinlock_wait_cycles":111958656,"thread_start_us":166277,"threads_started":2,"wal-append.queue_time_us":147802}
I20260709 13:17:01.809422 12064 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
W20260709 13:17:01.811231 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 117310us
I20260709 13:17:01.811957 17032 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.813310 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:01.816013 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:01.816804 17032 raft_consensus.cc:2764] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:17:01.819082 16936 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:01.822896 12062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35176 (request call id 994) took 3981 ms (client timeout 1881 ms). Trace:
W20260709 13:17:01.823460 12062 rpcz_store.cc:269] 0709 13:16:57.841343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.842877 (+  1534us) service_pool.cc:224] Handling call
0709 13:17:01.822465 (+3979588us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b540034b554. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:17:01.822648 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5254918016}
I20260709 13:17:01.825361 16712 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:01.826107 16712 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:01.819111 12032 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:01.804541 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.850879 11512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36660 (request call id 2663) took 2157 ms. Trace:
I20260709 13:17:01.851538 11512 rpcz_store.cc:276] 0709 13:16:59.688263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.689008 (+   745us) service_pool.cc:224] Handling call
0709 13:17:01.800951 (+2111943us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b5400371254. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:17:01.845354 (+ 44403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1961661696}
I20260709 13:17:01.843318 16712 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:01.862241 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:01.854558 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:01.867599 16874 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.851334 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:01.868568 17008 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.869576 11367 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:01.869984 17008 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:01.870325 16874 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.871778 16578 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:01.867295 11513 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:17:01.873265 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:01.876219 12034 leader_election.cc:400] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:17:01.876603 12034 leader_election.cc:403] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:17:01.878338 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 59485.1us
I20260709 13:17:01.879707 16811 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:01.881161 17057 raft_consensus.cc:3060] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:01.862185 11335 leader_election.cc:304] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:01.886443 11333 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:17:01.887436 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 80885.7us
I20260709 13:17:01.887758 16774 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:01.888191 11510 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:17:01.888468 16774 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:01.892275 16811 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:01.897217 16774 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:01.898267 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:01.899081 11364 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:01.901646 16794 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.903925 16794 raft_consensus.cc:2749] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:01.904844 11335 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:01.909439 11335 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:01.913337 17081 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.914197 17081 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:01.909348 16954 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:01.915524 16844 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.916272 11408 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:01.917527 17094 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:01.918377 17094 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.918831 17094 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:01.923923 11333 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:01.922917 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:01.927532 16806 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.929046 11438 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:01.914310 17093 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:01.930056 17093 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.930979 12067 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:17:01.932372 17093 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.933176 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:01.933003 16806 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.933992 12065 raft_consensus.cc:2368] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:17:01.936157 16806 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.917636 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:01.936951 16952 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:01.938870 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:01.939559 16811 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:01.942595 17099 log.cc:927] Time spent T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.904s	user 0.000s	sys 0.015s
I20260709 13:17:01.925068 11437 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:01.944392 16578 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:01.944948 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37092 (request call id 2112) took 1004 ms. Trace:
I20260709 13:17:01.945372 11366 rpcz_store.cc:276] 0709 13:17:00.940772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.940891 (+   119us) service_pool.cc:224] Handling call
0709 13:17:00.941200 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.557504 (+616304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.944360 (+386856us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b5400249854. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:01.944725 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415658368}
I20260709 13:17:01.923826 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:01.947865 11442 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:01.947918 16578 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:01.928891 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:01.963511 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:01.967646 16931 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:01.970232 16966 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:01.970556 16250 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:01.970887 16892 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.971531 16892 consensus_queue.cc:1059] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:01.970110 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:01.970278 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 61037.4us
I20260709 13:17:01.976790 16811 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:01.963415 17075 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:01.989957 11445 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:01.983430 11367 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:01.936257 17081 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:01.988787 11442 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:01.985783 11334 leader_election.cc:400] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:17:02.004298 11334 leader_election.cc:403] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:17:01.989693 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:02.008037 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 79468.9us
I20260709 13:17:01.979502 16966 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.036325 11409 leader_election.cc:400] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:02.036674 16966 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:02.036695 11409 leader_election.cc:403] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:17:02.012395 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.013103 16742 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:02.018944 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.019645 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:02.019762 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:02.020539 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:02.023687 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 84987.4us
I20260709 13:17:02.025092 16952 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.025538 17081 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:02.026057 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 102296us
I20260709 13:17:02.027704 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.029366 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:02.042085 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 56563us
I20260709 13:17:02.029546 11335 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:02.031515 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.034003 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.023458 16532 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:02.044642 11513 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:17:02.044646 16952 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.038270 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.037551 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.047029 11369 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:02.048223 17093 consensus_queue.cc:1059] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:02.035130 17094 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.049309 17093 raft_consensus.cc:3055] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:02.049678 17093 raft_consensus.cc:740] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:02.049786 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.036909 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.050505 12062 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:02.051203 17074 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15.
I20260709 13:17:02.050885 17093 consensus_queue.cc:260] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.051928 17094 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.052399 17093 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:02.052856 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.050909 11363 raft_consensus.cc:2393] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:02.034458 17075 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:02.036211 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.046456 16806 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:02.058912 11364 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:17:02.055550 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.044762 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.056097 11368 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:02.034832 17057 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:17:02.053598 11370 raft_consensus.cc:2368] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:17:02.048305 11409 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:02.060871 11334 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.023586 16578 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.064574 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:02.065805 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46224.1us
I20260709 13:17:02.066293 11362 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:02.068856 12067 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:02.041023 17081 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:02.048310 16952 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.076952 16967 raft_consensus.cc:2804] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:02.077656 16967 raft_consensus.cc:697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:02.078017 12060 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:02.078706 16967 consensus_queue.cc:237] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.080740 11365 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:02.082070 11362 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:02.079535 11516 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:02.081319 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:02.079149 16785 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.074424 17081 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:02.094383 11364 raft_consensus.cc:2393] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 16: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:02.079672 16899 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:02.091683 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 62029.5us
I20260709 13:17:02.096549 11437 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:02.096539 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:02.078035 16250 raft_consensus.cc:3055] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:02.102756 16250 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:02.103854 16250 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.104679 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.105211 16250 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:02.107683 16785 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.108932 11369 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11.
I20260709 13:17:02.109861 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.110023 16785 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:02.115535 16843 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:02.115823 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.118126 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.120838 16742 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.119220 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.121779 16899 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:02.122402 16899 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:02.122835 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.124919 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.126730 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.131609 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:02.132038 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:02.132872 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 67373.7us
I20260709 13:17:02.105497 11363 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:17:02.134848 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.135895 11363 raft_consensus.cc:2393] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:02.136912 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.138005 11363 raft_consensus.cc:2410] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:17:02.138985 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.139596 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:02.139717 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 58490us
I20260709 13:17:02.133075 11362 raft_consensus.cc:2393] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:02.166561 12062 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:02.167387 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.178886 12067 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:17:02.180001 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:02.207206 12060 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:17:02.167985 16806 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:17:02.211782 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44404.3us
I20260709 13:17:02.123303 16742 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.213174 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:17:02.214252 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 46179.3us
I20260709 13:17:02.217320 11361 raft_consensus.cc:2410] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:17:02.218055 11370 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:17:02.228645 11335 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:02.230774 11366 raft_consensus.cc:1240] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:17:02.231629 16785 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:02.181113 17057 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:02.235455 11365 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:02.215512 16806 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.248329 11368 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:02.254150 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.250895 16806 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:02.281230 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.282094 12067 raft_consensus.cc:2368] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:17:02.285396 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.286237 12062 raft_consensus.cc:2368] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:02.244673 17008 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:17:02.289201 12060 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:02.291450 11335 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.301556 12033 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:02.303275 17122 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:17:02.310243 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.310564 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.311117 12067 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:02.311547 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.312055 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.312708 12061 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:02.316831 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.321530 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.323740 17055 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:02.324229 17055 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:02.324615 17055 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:02.325156 11334 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:02.326210 11334 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:02.323225 16952 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:17:02.327231 16786 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:02.292658 16931 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:02.265399 16843 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:17:02.333278 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.326462 16532 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:02.334746 16250 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:02.338748 12064 raft_consensus.cc:1240] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:02.315384 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:02.342386 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44656.2us
I20260709 13:17:02.343036 16952 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:02.343364 12033 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:02.345995 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 255025us
I20260709 13:17:02.341692 16846 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:17:02.365048 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.371505 16250 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:02.345615 16532 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:02.345701 16786 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
W20260709 13:17:02.380107 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 284067us
I20260709 13:17:02.382450 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.383232 12065 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:17:02.368439 17075 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:02.400147 12063 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:17:02.407284 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.408106 12065 raft_consensus.cc:2368] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:02.425201 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.425662 11409 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:02.425794 16786 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.426729 16756 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:02.427134 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:02.429656 17094 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:02.430212 17094 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.430584 17094 raft_consensus.cc:3060] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:02.435717 17080 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:02.377485 16899 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.438261 17094 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:02.429642 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 86375.8us
W20260709 13:17:02.435717 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 141776us
W20260709 13:17:02.443670 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 111415us
W20260709 13:17:02.430665 17123 log.cc:927] Time spent T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:02.448028 11513 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:02.430819 16532 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.483937 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:17:02.489723 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 225500us
W20260709 13:17:02.491459 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 94095us
I20260709 13:17:02.504092 12033 leader_election.cc:400] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 12. Current term is 13.
I20260709 13:17:02.505594 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.513695 11516 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:02.527267 11335 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.529245 16926 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:17:02.530911 16803 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:02.531526 16803 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.534111 16803 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.535559 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.542768 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:02.543007 16967 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.543737 11515 raft_consensus.cc:2393] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:02.546060 12061 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:02.545910 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.546844 12062 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:02.547379 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:17:02.551544 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 66066.8us
I20260709 13:17:02.553383 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.589157 16931 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.589746 16931 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:02.595479 16806 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.596978 17143 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.591650 16774 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:02.546818 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.602875 17143 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:02.604005 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 96282.7us
I20260709 13:17:02.605494 17143 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:02.599291 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.606379 12060 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
W20260709 13:17:02.609851 17126 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.266s	user 0.002s	sys 0.000s
W20260709 13:17:02.610420 16966 consensus_meta.cc:287] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.005s
I20260709 13:17:02.610854 16966 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:02.611625 17093 consensus_meta.cc:287] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.006s
I20260709 13:17:02.612484 17093 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.617873 17093 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.613054 16966 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:02.615581 16806 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:02.616070 17081 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.601807 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:02.620972 17134 log.cc:927] Time spent T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:17:02.625618 12064 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 9.
I20260709 13:17:02.628345 16899 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.632192 12065 raft_consensus.cc:2468] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
W20260709 13:17:02.630666 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 203598us
I20260709 13:17:02.641785 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.647037 16966 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.648836 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.651973 16467 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.652952 11512 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:02.654016 11439 raft_consensus.cc:1240] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:17:02.543749 16967 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.655498 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:02.656311 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:02.657264 17137 log.cc:927] Time spent T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 13:17:02.659775 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 60564.3us
W20260709 13:17:02.662444 17135 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:17:02.663625 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.656204 17094 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.676591 11441 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:02.677179 16250 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:17:02.676208 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 184118us
I20260709 13:17:02.665239 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.658874 17055 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.666286 17074 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:17:02.673851 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.680035 11438 raft_consensus.cc:2393] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:02.679068 11516 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:02.680446 11334 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:02.666167 17093 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.682315 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.634662 16756 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:02.688651 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:02.690380 11409 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.683346 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:02.692085 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.694069 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.694681 16952 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.695250 16952 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.696215 11370 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:02.697497 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.699844 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.704551 17081 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.705492 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6afc1298d03c4fa6a7ce8e41231e8cf4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.705806 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.706630 11511 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:02.709875 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.710698 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.711517 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.713431 12068 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:02.713747 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.715041 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:02.715844 12068 raft_consensus.cc:3055] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:02.716202 12068 raft_consensus.cc:740] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:02.716953 12069 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:02.717374 12068 consensus_queue.cc:260] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.710693 16786 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:02.718504 16786 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:02.718763 12068 raft_consensus.cc:3060] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:02.720783 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:02.721792 16786 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.723672 16756 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.725008 16967 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:02.727947 11436 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:02.728826 11367 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:17:02.729269 11367 pending_rounds.cc:85] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 8
I20260709 13:17:02.729521 11367 pending_rounds.cc:107] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:17:02.729806 11367 raft_consensus.cc:2889] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.666980 17012 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:17:02.731388 16467 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.732710 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.733585 12060 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:02.733785 16467 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:02.734056 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:02.734584 16806 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:02.735368 12060 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:02.736429 17094 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.666980 16794 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:17:02.737718 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.738302 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.739385 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.739289 11362 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:02.687951 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.694013 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.742861 11335 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.746999 17094 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.748302 12068 raft_consensus.cc:1275] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:02.749112 16578 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:02.743155 11515 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:02.750429 12069 raft_consensus.cc:2410] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:17:02.747622 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:02.728987 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:02.749883 16954 consensus_queue.cc:1059] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:17:02.707182 16774 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:02.697557 16952 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:02.676522 16966 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.756558 16844 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:02.749815 16803 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:17:02.776922 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 86006.7us
I20260709 13:17:02.784523 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.785086 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:02.777194 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:17:02.787765 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 53771.9us
I20260709 13:17:02.776400 17080 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:02.800539 11515 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:02.794104 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:02.802479 11409 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.803148 16806 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:02.803830 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.804634 16806 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:02.805840 11409 leader_election.cc:400] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:02.808944 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.790705 11361 raft_consensus.cc:2468] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:17:02.810158 11515 raft_consensus.cc:2468] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:02.810815 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.812100 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:02.813290 11436 raft_consensus.cc:2410] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:17:02.814065 12032 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:02.815248 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 134834us
I20260709 13:17:02.816680 11333 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.819903 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:02.827638 11511 raft_consensus.cc:1275] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:02.819274 16967 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:02.833572 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:02.834412 11438 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:02.829937 17144 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:02.836594 16967 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:02.837073 16967 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.837447 16967 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:02.838873 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:02.838927 11439 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:02.833925 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:02.842621 16874 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:02.846922 11515 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:02.772431 16774 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.867053 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:02.776100 16931 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:02.821949 17081 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:02.824249 16952 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:02.884652 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:02.868875 16811 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:02.800691 17080 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:02.820535 16806 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:17:02.886919 17144 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:02.880896 17008 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 13:17:02.926906 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 107040us
I20260709 13:17:02.917624 11483 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:17:02.952277 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 140232us
I20260709 13:17:02.894316 17012 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:17:02.898967 17094 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:02.966667 16250 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:02.983317 16808 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:02.983646 12032 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:02.983923 17149 log.cc:927] Time spent T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:17:02.952570 17085 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:02.957734 17144 raft_consensus.cc:3060] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:02.998183 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 159369us
I20260709 13:17:02.957266 16532 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:17:02.961617 16467 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:02.913839 17055 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:03.004287 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.005084 12061 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:03.000996 16250 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.007262 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.003670 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:03.009989 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.009650 17094 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.013196 17094 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.015823 17094 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:02.979807 16954 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.018661 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.003674 16594 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:03.021021 11408 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:03.004240 16467 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:03.017185 16954 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.025938 16774 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:03.074390 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 166579us
W20260709 13:17:03.101697 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 83114.7us
I20260709 13:17:03.012912 16966 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:03.022608 16594 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:03.024780 16467 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:03.103790 17122 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:17:03.156375 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.103998 17093 raft_consensus.cc:2804] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:03.154489 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.153980 16931 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:03.105995 17085 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.106832 17081 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:03.199872 17157 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.208s	user 0.004s	sys 0.000s
W20260709 13:17:03.200047 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 206003us
W20260709 13:17:03.201095 17151 log.cc:927] Time spent T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:17:03.109091 17039 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.112102 12033 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:03.204448 16250 consensus_queue.cc:1048] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:17:03.206064 12032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.203990 16843 raft_consensus.cc:3055] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:03.259697 16843 raft_consensus.cc:740] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:03.260684 16843 consensus_queue.cc:260] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.262012 16843 raft_consensus.cc:3060] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:03.255707 11409 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:03.266129 17160 log.cc:927] Time spent T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:17:03.275494 17080 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:03.279465 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:03.280364 12067 raft_consensus.cc:2393] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 11: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:17:03.282083 17155 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
I20260709 13:17:03.211635 17093 raft_consensus.cc:697] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:03.284998 16594 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.285812 17093 consensus_queue.cc:237] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:03.286880 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 268351us
I20260709 13:17:03.288059 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
W20260709 13:17:03.289915 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 85481.7us
I20260709 13:17:03.289515 16967 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.293613 17085 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.309545 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.310698 17144 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.312820 17144 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:03.313755 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 101151us
I20260709 13:17:03.314621 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.316995 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.320011 16967 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:03.325680 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.327370 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.328219 16874 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.329067 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.331420 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:03.332093 12064 raft_consensus.cc:2368] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:03.332854 16954 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:03.334827 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:03.337074 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:03.339250 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:03.339920 11361 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:17:03.340269 11483 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.344067 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:03.300114 17081 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.346115 16874 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.344187 11483 leader_election.cc:304] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.353283 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:03.353598 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:03.355484 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:03.356176 11514 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:03.356086 11409 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.358639 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:03.358613 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:03.359458 11362 raft_consensus.cc:2393] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:03.362313 11409 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.364754 11409 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.366132 11369 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:03.357373 16874 raft_consensus.cc:2749] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.369896 17169 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:03.370716 17169 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:03.371084 17169 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:03.292909 17039 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.365603 17146 raft_consensus.cc:2804] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:17:03.379751 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 83236.7us
I20260709 13:17:03.382292 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:03.383106 12060 raft_consensus.cc:2368] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:17:03.357563 17158 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.363816 17085 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.371526 17169 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:03.385106 17146 raft_consensus.cc:697] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:03.396390 12067 raft_consensus.cc:2468] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:03.397513 11437 raft_consensus.cc:1240] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:03.396486 11409 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:03.405668 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 61551.2us
I20260709 13:17:03.406673 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:03.410741 11444 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:17:03.415195 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.418838 16838 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:03.420420 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:03.416386 16803 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.423969 16803 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.426075 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.426856 12067 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:17:03.427428 17159 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.428012 17159 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.430143 17159 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:03.401911 16786 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.432156 16786 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.433614 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41514 (request call id 1269) took 1305 ms. Trace:
I20260709 13:17:03.434379 16786 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:03.436971 11409 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.433987 11441 rpcz_store.cc:276] 0709 13:17:02.127950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.131135 (+  3185us) service_pool.cc:224] Handling call
0709 13:17:02.131512 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.677263 (+545751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.433403 (+756140us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54001b6cd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:03.433558 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253155072}
I20260709 13:17:03.442312 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.443158 12065 raft_consensus.cc:2410] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:17:03.443138 12033 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.442488 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.445317 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:03.445256 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.446489 12067 raft_consensus.cc:2410] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:17:03.445628 17173 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:03.448405 17173 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.448715 17173 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:03.457883 11437 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:17:03.458559 16811 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:03.458703 17144 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:17:03.458925 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.443089 16806 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.465989 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.466847 16844 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:03.467533 16811 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:03.467726 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:03.469379 16786 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:03.469741 16786 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.470062 16786 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:03.442746 16774 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.472060 16774 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.472908 12065 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:03.474416 16774 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:03.474138 17146 consensus_queue.cc:237] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.475665 11362 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:03.476727 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.414065 17158 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.478663 16774 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:03.479746 11445 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:17:03.472069 11439 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:17:03.480685 11439 pending_rounds.cc:85] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 11
I20260709 13:17:03.480888 11439 pending_rounds.cc:107] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:17:03.481122 11439 raft_consensus.cc:2889] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:03.481577 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:03.495955 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:03.383852 16467 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.498986 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.491638 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.501926 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:03.426082 16803 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:03.508277 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
W20260709 13:17:03.513757 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 72180.3us
I20260709 13:17:03.514565 11483 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.516928 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.517373 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.507602 11518 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:17:03.517956 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:03.518586 11439 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
W20260709 13:17:03.519960 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 76893.5us
I20260709 13:17:03.522564 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.524068 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:03.482333 17158 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:03.525090 11511 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:03.525200 16874 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:03.526170 16578 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:03.490769 17159 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:03.524669 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.510187 16954 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:03.538291 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.540388 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:03.543083 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:03.521502 16806 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.458599 17173 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.528061 17085 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:03.560014 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:03.545544 16844 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:17:03.563377 17168 consensus_queue.cc:1059] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:17:03.550693 16774 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:03.564504 16774 raft_consensus.cc:3055] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:03.564890 16774 raft_consensus.cc:740] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:03.565755 16774 consensus_queue.cc:260] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.566949 16774 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:03.567507 17048 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:03.568078 17048 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.570225 17048 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:03.573237 17164 log.cc:927] Time spent T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.581s	user 0.002s	sys 0.000s
I20260709 13:17:03.583604 11518 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:17:03.556399 16803 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:03.559754 16846 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:03.564531 17148 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:03.554527 17159 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.608261 12060 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:17:03.613919 11361 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:03.591004 16803 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.616753 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:03.618706 11369 raft_consensus.cc:2376] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 11. Re-sending same reply.
I20260709 13:17:03.621814 11514 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:03.622231 11366 raft_consensus.cc:1240] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:03.622586 11519 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:17:03.623216 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:03.606482 17148 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:03.581241 16532 consensus_queue.cc:1175] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "34a42aa6aa0a4778a532c6433d70b981" responder_term: 9 status { last_received { term: 9 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 9 index: 9 } }
W20260709 13:17:03.634450 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 117152us
I20260709 13:17:03.605063 17085 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:17:03.639757 11518 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:03.619480 16803 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:17:03.643504 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 186906us
I20260709 13:17:03.644840 17170 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:03.645309 17170 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:03.645722 17170 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:03.646003 17150 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.662s	user 0.003s	sys 0.000s
I20260709 13:17:03.644376 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.623610 17039 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:03.633373 17148 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:03.660795 11510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2707) took 1027 ms. Trace:
I20260709 13:17:03.661314 11510 rpcz_store.cc:276] 0709 13:17:02.633858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.633967 (+   109us) service_pool.cc:224] Handling call
0709 13:17:02.657064 (+ 23097us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:02.657538 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:02.657565 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:02.658131 (+   566us) log.cc:844] Serialized 52 byte log entry
0709 13:17:02.911020 (+252889us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:02.911062 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.911130 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:03.645345 (+734215us) spinlock_profiling.cc:243] Waited 8.72 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:17:03.648876 (+  3531us) raft_consensus.cc:1685] finished
0709 13:17:03.648901 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.660696 (+ 11795us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1716,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":20061056,"thread_start_us":252481,"threads_started":1,"wal-append.queue_time_us":269994}
I20260709 13:17:03.679749 17142 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:03.680281 17142 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:03.681322 17142 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.682757 17142 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:03.689226 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 143902us
I20260709 13:17:03.706056 11335 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:03.686800 17175 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:17:03.675284 16756 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:03.712220 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 67933.7us
I20260709 13:17:03.687047 16794 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:03.719820 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:03.716017 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.655841 16811 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.763760 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:03.850335 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 134378us
I20260709 13:17:03.675032 16846 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 13:17:03.853726 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 145930us
I20260709 13:17:03.851229 11409 leader_election.cc:400] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 12. Current term is 13.
I20260709 13:17:03.716652 16954 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:03.859082 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 139468us
I20260709 13:17:03.720055 17048 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:03.709136 17055 consensus_queue.cc:1059] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.141s
I20260709 13:17:03.709489 17074 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.140s
I20260709 13:17:03.878331 17191 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.878948 17191 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.716552 17039 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.147s
I20260709 13:17:03.854827 17173 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:03.880178 17012 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:17:03.858882 16756 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:03.889680 17057 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:17:03.862818 17080 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:03.851737 16794 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:03.894578 16594 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:03.889374 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.888779 11440 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:17:03.895653 17174 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
W20260709 13:17:03.895493 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44415.1us
I20260709 13:17:03.901510 11409 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:03.903091 11409 leader_election.cc:400] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:03.888942 17063 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.239s
I20260709 13:17:03.915206 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:03.921561 17194 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:03.892656 17187 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:03.889001 17170 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:03.932184 17176 log.cc:927] Time spent T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:17:03.889521 16881 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.214s
I20260709 13:17:03.934504 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:03.935071 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.938048 17074 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:03.939363 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:03.940120 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:03.942399 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 406240us
I20260709 13:17:03.952428 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:03.960748 16578 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.913508 17159 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:03.931429 12065 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:03.972687 11484 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:03.916224 16756 raft_consensus.cc:2804] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:17:03.975561 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 76575.8us
I20260709 13:17:03.962966 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:03.978863 16578 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } attrs { replace: true } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.888095 17048 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:03.984298 11363 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:17:04.017791 11363 pending_rounds.cc:85] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 9
I20260709 13:17:04.018091 11363 pending_rounds.cc:107] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:17:04.018306 11363 raft_consensus.cc:2889] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:04.018723 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46155.4us
I20260709 13:17:03.974144 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:03.918442 16844 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:03.927834 17168 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:03.969394 17180 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:17:04.029211 16250 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:17:04.037030 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 134057us
I20260709 13:17:04.036712 16844 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.057039 16578 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.058002 16794 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.059223 17183 raft_consensus.cc:3055] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:04.059654 17183 raft_consensus.cc:740] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:04.060484 17183 consensus_queue.cc:260] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.061511 16467 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.061815 17183 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:03.984407 17144 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:17:04.066649 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 78140.7us
I20260709 13:17:04.061645 11440 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:04.054183 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 91317.9us
I20260709 13:17:04.049546 17201 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:04.016203 16756 raft_consensus.cc:697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:17:04.070717 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 50910.1us
W20260709 13:17:04.074649 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.030058 16846 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:04.090572 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:04.094271 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:04.090798 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.094332 16250 consensus_queue.cc:1059] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:04.070220 17144 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:04.099298 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.055032 16844 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.097126 11437 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 12: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:04.105938 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.045683 17168 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.103828 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.109349 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.112336 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:04.039523 17187 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.129799 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.138325 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.139950 17048 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.107846 17085 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:17:04.140662 17200 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:04.129829 16794 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.148710 16794 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.150180 17173 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:04.151750 17173 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:04.153285 11438 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:17:04.155690 17191 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.157445 17168 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:04.130903 16786 consensus_meta.cc:287] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.001s
I20260709 13:17:04.158236 16786 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.160403 16786 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.130358 17057 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:17:04.151187 17169 consensus_meta.cc:287] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.711s	user 0.004s	sys 0.004s
W20260709 13:17:04.164080 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 69895.2us
I20260709 13:17:04.120502 17193 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:04.164178 11361 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:04.180116 16594 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.180751 16594 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.188370 12063 raft_consensus.cc:1240] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:04.189484 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:04.189641 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:04.190766 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2eaec0f849964172b0dcfba0af0e7fa3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:04.191337 11370 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:04.191578 11369 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:17:04.191987 11369 pending_rounds.cc:85] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Aborting all ops after (but not including) 8
I20260709 13:17:04.192219 11369 pending_rounds.cc:107] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:17:04.192462 11369 raft_consensus.cc:2889] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:04.197516 11510 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:04.198483 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.199766 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.201025 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.202281 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:04.203950 17206 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:04.204572 17206 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.211273 16594 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:17:04.213716 12065 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:04.224974 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:04.225692 12068 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:17:04.219708 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.226748 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:04.227433 12066 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:17:04.227720 16786 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:04.228439 12069 raft_consensus.cc:1240] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:04.220389 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.227428 11333 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.222671 17159 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.220290 16844 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:17:04.175913 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 99236.5us
I20260709 13:17:04.223819 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:04.213851 17205 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:04.136349 16756 consensus_queue.cc:237] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.176409 17200 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:04.232335 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:04.233436 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 95165.2us
I20260709 13:17:04.237200 11370 raft_consensus.cc:2468] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 17.
I20260709 13:17:04.250716 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:04.253269 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:04.256603 11441 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 16: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:04.228606 16811 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.255364 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.264220 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:04.264511 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.249454 17205 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:04.267057 17205 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.176872 17169 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.269284 11438 raft_consensus.cc:1240] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:04.268936 17205 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:04.270305 17169 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.275884 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:04.276520 12062 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:04.285084 16803 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.004s
I20260709 13:17:04.285562 16803 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.287505 16846 consensus_queue.cc:1048] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:04.259671 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:04.294791 11334 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.283175 12062 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:04.297361 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:04.257630 17012 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:17:04.255918 16954 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:17:04.300374 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 44410.1us
I20260709 13:17:04.258199 16774 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:04.173560 17207 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:17:04.305168 17200 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:04.313053 17210 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:04.314463 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:04.305584 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:04.316762 11441 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:04.317198 12061 raft_consensus.cc:1275] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:04.303776 11334 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:04.255281 16785 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:17:04.304201 17039 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:04.309623 17194 consensus_queue.cc:1059] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:04.257402 16844 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.310930 17187 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.254673 17193 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:17:04.260478 17055 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:17:04.328768 12061 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:17:04.331641 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:04.335119 11409 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:04.300089 17168 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:04.255657 17159 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.306056 16786 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:04.307240 17075 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:04.264108 16467 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
W20260709 13:17:04.383489 17142 consensus_meta.cc:287] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.700s	user 0.014s	sys 0.000s
I20260709 13:17:04.386049 17144 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.336902 17008 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:04.347615 17215 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:04.356487 16844 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.357018 11441 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 18.
I20260709 13:17:04.307495 16954 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:04.399039 16954 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:04.344967 16803 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.344601 17169 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:17:04.342592 17194 raft_consensus.cc:3055] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:04.401559 17194 raft_consensus.cc:740] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:04.362751 17158 raft_consensus.cc:2804] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:04.363277 17207 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:04.402501 17194 consensus_queue.cc:260] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.403437 17207 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.403883 17194 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:04.396270 17215 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:04.405792 17207 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.405958 17215 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.408740 17159 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:04.366532 11254 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6e87b213a4554d94825aceec352fc094 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:17:04.410723 11254 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 44382.7us
I20260709 13:17:04.367614 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.373535 17193 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:04.341609 17213 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:04.407267 17215 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:04.343104 17168 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.417687 11363 raft_consensus.cc:1240] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:17:04.421948 12060 raft_consensus.cc:1240] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:17:04.397294 16786 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.402575 17158 raft_consensus.cc:697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 17 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:17:04.427593 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 119625us
W20260709 13:17:04.447788 11408 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.453572 11333 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:04.419415 17168 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:04.425787 17214 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.115s	user 0.003s	sys 0.000s
I20260709 13:17:04.474174 11363 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:17:04.425518 16786 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:04.475603 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:04.423076 17008 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:04.415643 17193 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.482431 16594 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.483768 17218 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.482111 16785 raft_consensus.cc:493] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:17:04.494601 17148 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.000s
W20260709 13:17:04.501536 11485 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.503872 17008 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.507834 17144 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.500164 17048 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:04.510691 17217 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:17:04.509217 11444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2806) took 1091 ms. Trace:
I20260709 13:17:04.509938 17193 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:04.496308 17142 consensus_queue.cc:1059] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:17:04.528879 17142 raft_consensus.cc:3055] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:04.529235 17142 raft_consensus.cc:740] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:04.528092 11444 rpcz_store.cc:276] 0709 13:17:03.417572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.417739 (+   167us) service_pool.cc:224] Handling call
0709 13:17:03.418047 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:03.436965 (+ 18918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:03.436995 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:03.441310 (+  4315us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.253617 (+812307us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b1800461ec0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b9910c02 00007f35bc59820b 00007f35bc59dd23 00007f35bc59e0d1 00007f35bc9b9a0f
0709 13:17:04.306832 (+ 53215us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:04.344286 (+ 37454us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.344575 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.344649 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:04.508930 (+164281us) raft_consensus.cc:1685] finished
0709 13:17:04.509025 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:04.509159 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21546,"prepare.queue_time_us":34883,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":36441,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":47859,"spinlock_wait_cycles":7312256,"thread_start_us":53033,"threads_started":1,"wal-append.queue_time_us":937336}
I20260709 13:17:04.529979 17142 consensus_queue.cc:260] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.530491 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:04.510582 16786 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.528195 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:04.531138 17142 raft_consensus.cc:3060] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:04.536944 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:04.538239 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:04.542680 17168 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.544858 16786 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.523903 17148 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.550868 16794 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.532804 16785 raft_consensus.cc:515] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.524266 16803 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:04.574581 11367 raft_consensus.cc:1240] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:04.576773 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:04.577181 17168 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.584280 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:04.520855 16594 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.589844 11513 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:04.555413 16954 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.604228 11438 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:04.608995 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.609699 12064 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
W20260709 13:17:04.613456 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 159966us
I20260709 13:17:04.614277 11333 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:04.669337 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55155.1us
I20260709 13:17:04.531591 17218 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.673202 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:04.679351 11517 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:04.684774 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.558154 17158 consensus_queue.cc:237] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.588701 16803 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.688702 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.717115 11511 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
W20260709 13:17:04.692029 17219 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:17:04.692595 17144 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:04.705765 16785 leader_election.cc:290] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.706300 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.734683 12065 raft_consensus.cc:1275] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:17:04.740330 17220 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:17:04.598812 11362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:17:04.760838 11362 raft_consensus.cc:606] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:04.761194 11362 raft_consensus.cc:614] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:04.743721 16594 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.678570 16794 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.759500 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:04.773943 11519 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:04.758507 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.668651 17215 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:17:04.766204 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:04.776273 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:04.777096 11515 raft_consensus.cc:2393] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:04.778038 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.778556 11515 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:04.779363 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.771487 17170 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.787704 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:04.805855 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44cbd38458af4b03b0d3f362bd6a2f85" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.779484 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:04.812029 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:04.813338 11514 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:04.822137 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:04.827375 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46206.9us
I20260709 13:17:04.822887 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:04.832913 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:04.833735 11510 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 16: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:04.752833 17144 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.810168 17075 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:04.823982 17209 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:04.844614 11483 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:04.847905 17209 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.908699 17209 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:04.915829 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.839370 17210 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:04.920148 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:04.816632 17213 consensus_queue.cc:1048] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:17:04.924155 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:04.816268 16794 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.818240 17169 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:17:04.842579 17146 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:04.829358 16756 consensus_queue.cc:1048] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:17:04.934504 17074 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.938918 17074 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:04.939554 17222 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:04.940029 17222 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.940363 17222 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:04.941169 17074 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.943516 17225 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:04.944044 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.944736 11442 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 10.
I20260709 13:17:04.949189 12066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:04.949779 12066 raft_consensus.cc:606] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:04.923126 16467 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:04.920601 17148 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:04.950130 12066 raft_consensus.cc:614] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:04.925872 17210 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.849181 17075 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.942292 17206 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:04.934419 17231 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:04.945822 16756 consensus_queue.cc:1243] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:17:04.986956 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:04.990959 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:04.960501 17210 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:04.955158 17207 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:04.988065 17057 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:05.026443 17057 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.981331 16594 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:05.043836 16594 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.046773 16594 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.054549 17209 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.057307 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:05.058755 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:05.010306 11519 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:17:05.023366 17170 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:05.023970 17240 log.cc:927] Time spent T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:17:05.024721 17234 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:17:05.062250 11512 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:17:04.985502 16954 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:04.944085 17225 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:04.956902 17146 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:05.059358 17241 log.cc:927] Time spent T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:17:05.106765 17170 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.151806 11363 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:05.123382 17218 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:05.156464 17243 log.cc:927] Time spent T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:17:05.161795 17213 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:05.163825 16794 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.164618 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.165937 17225 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.172717 17146 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.173238 17085 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.177772 11517 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 18.
I20260709 13:17:05.154440 17189 raft_consensus.cc:3055] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:05.085373 17207 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.164604 16756 raft_consensus.cc:493] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:05.117681 17191 consensus_queue.cc:1059] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 13:17:05.182173 17229 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.173s	user 0.002s	sys 0.000s
W20260709 13:17:05.187530 11262 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39444 (request call id 354) took 20008 ms (client timeout 15000 ms). Trace:
I20260709 13:17:05.198259 17057 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.203533 17222 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.205149 11335 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:05.321154 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 116121us
I20260709 13:17:05.322355 17222 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.323014 16794 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:05.327991 11362 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 17: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:05.224431 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:05.330134 16756 raft_consensus.cc:515] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.225867 16774 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:05.226043 11268 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 8 to 12, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:05.228503 17170 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.225965 11333 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:05.331791 16774 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:17:05.332116 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 106225us
I20260709 13:17:05.332717 16756 leader_election.cc:290] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.333225 11333 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:17:05.230036 11361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.025s	sys 0.052s
I20260709 13:17:05.238819 11367 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:17:05.258035 16954 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:05.259341 17253 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:05.262185 17235 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.146s	user 0.005s	sys 0.000s
W20260709 13:17:05.262926 17246 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.058s	user 0.002s	sys 0.002s
I20260709 13:17:05.266883 16803 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:05.267663 17247 log.cc:927] Time spent T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.057s	user 0.006s	sys 0.006s
I20260709 13:17:05.268137 11368 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:05.269289 17144 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.096293 17209 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:05.275733 17227 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.337s	user 0.005s	sys 0.000s
W20260709 13:17:05.276131 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 168046us
I20260709 13:17:05.284173 12032 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:05.404498 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 120429us
I20260709 13:17:05.288798 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.298961 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:05.405961 11510 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:05.300547 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.303280 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.304041 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.406945 11515 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:17:05.305862 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.408365 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.306522 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.307166 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:05.307864 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.308651 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:05.310593 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.409572 11519 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 9.
I20260709 13:17:05.410550 12061 raft_consensus.cc:2368] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:05.403889 12034 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:05.313331 16881 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.354911 17225 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:05.223043 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:05.334239 11361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.025s	sys 0.052s
W20260709 13:17:05.287349 17192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.004s	sys 0.028s
I20260709 13:17:05.332101 11268 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:05.370687 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:05.371956 17075 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.350523 16532 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:05.283170 17210 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.387511 17222 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
W20260709 13:17:05.317108 11262 rpcz_store.cc:269] 0709 13:16:45.179017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.179214 (+   197us) service_pool.cc:224] Handling call
0709 13:17:05.187209 (+20007995us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:58.951630 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:59.179819 (+228189us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:59.179881 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:16:59.179901 (+    20us) tablet.cc:662] Decoding operations
0709 13:16:59.181703 (+  1802us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:59.181824 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:59.181849 (+    25us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:16:59.190762 (+  8913us) tablet.cc:701] Row locks acquired
0709 13:16:59.190817 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:16:59.190982 (+   165us) write_op.cc:270] Start()
0709 13:16:59.191156 (+   174us) write_op.cc:276] Timestamp: P: 1783603019190921 usec, L: 0
0709 13:16:59.191193 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:16:59.191929 (+   736us) log.cc:844] Serialized 8601 byte log entry
0709 13:17:01.420723 (+2228794us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:17:01.646765 (+226042us) op_driver.cc:464] REPLICATION: finished
0709 13:17:01.858928 (+212163us) write_op.cc:301] APPLY: starting
0709 13:17:01.859097 (+   169us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:02.229580 (+370483us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:02.229632 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:02.628790 (+399158us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:02.701506 (+ 72716us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:17:02.701573 (+    67us) write_op.cc:312] APPLY: finished
0709 13:17:02.702221 (+   648us) log.cc:844] Serialized 196 byte log entry
0709 13:17:02.824674 (+122453us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:02.898265 (+ 73591us) write_op.cc:454] Released schema lock
0709 13:17:02.898371 (+   106us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:02.898991 (+   620us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:02.899024 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":597021,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":210838,"apply.run_cpu_time_us":44850,"apply.run_wall_time_us":1040100,"cfile_cache_hit":90,"cfile_cache_hit_bytes":95999,"num_ops":22,"prepare.queue_time_us":353432,"prepare.run_cpu_time_us":49062,"prepare.run_wall_time_us":696938,"raft.queue_time_us":135644,"raft.run_cpu_time_us":46392,"raft.run_wall_time_us":238170,"replication_time_us":2447801,"spinlock_wait_cycles":3699456,"thread_start_us":1051414,"threads_started":5,"wal-append.queue_time_us":839719,"wal-append.run_cpu_time_us":31100,"wal-append.run_wall_time_us":3877119}]]}
I20260709 13:17:05.361129 11335 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:05.397487 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:05.332193 16774 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:05.291828 17189 raft_consensus.cc:740] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:17:05.432889 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 71832.2us
I20260709 13:17:05.412204 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.410884 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:17:05.315526 17233 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.576s	user 0.003s	sys 0.001s
I20260709 13:17:05.413820 11516 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:05.408368 11439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51484 (request call id 2816) took 1441 ms. Trace:
W20260709 13:17:05.422528 17192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.004s	sys 0.028s
I20260709 13:17:05.452636 16774 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.454092 16774 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:05.456825 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2766) took 1058 ms. Trace:
I20260709 13:17:05.457952 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:05.457217 11364 rpcz_store.cc:276] 0709 13:17:04.398134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.398374 (+   240us) service_pool.cc:224] Handling call
0709 13:17:04.409541 (+ 11167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.429194 (+ 19653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.429222 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:04.469109 (+ 39887us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.737708 (+268599us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:04.760844 (+ 23136us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.779912 (+ 19068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.779982 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:05.241332 (+461350us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:17:05.456515 (+215183us) raft_consensus.cc:1685] finished
0709 13:17:05.456621 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.456759 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92868,"prepare.queue_time_us":100551,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":7847,"raft.queue_time_us":88247,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":8434,"spinlock_wait_cycles":16450304,"thread_start_us":215025,"threads_started":1,"wal-append.queue_time_us":435286}
I20260709 13:17:05.459568 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:05.400223 16954 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.465241 17207 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:05.465981 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 51666us
I20260709 13:17:05.466130 11364 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:17:05.362622 17169 consensus_queue.cc:1059] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:17:05.467476 17169 raft_consensus.cc:3055] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:05.467844 17169 raft_consensus.cc:740] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:05.468609 17169 consensus_queue.cc:260] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.469763 17169 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:05.471673 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.473117 11370 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2764) took 1134 ms. Trace:
I20260709 13:17:05.473523 11370 rpcz_store.cc:276] 0709 13:17:04.338676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.339703 (+  1027us) service_pool.cc:224] Handling call
0709 13:17:04.340062 (+   359us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:04.488258 (+148196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.488290 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:04.515348 (+ 27058us) log.cc:844] Serialized 52 byte log entry
0709 13:17:04.732909 (+217561us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:17:04.753985 (+ 21076us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.778037 (+ 24052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.778117 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:05.240465 (+462348us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:17:05.472843 (+232378us) raft_consensus.cc:1685] finished
0709 13:17:05.472932 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.473061 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108744,"prepare.queue_time_us":34681,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":105761,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":28507,"spinlock_wait_cycles":5438208,"thread_start_us":268901,"threads_started":2,"wal-append.queue_time_us":451756}
I20260709 13:17:05.359987 17253 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.477607 17253 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.427088 17209 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:17:05.478785 17209 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:05.479110 17209 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:05.479943 17209 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.481108 17209 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:05.447055 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.425254 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:05.427817 11361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2763) took 1087 ms. Trace:
I20260709 13:17:05.289294 17213 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.426767 11268 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 13, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.488435 11510 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:05.434196 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:05.423761 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 53166us
I20260709 13:17:05.489452 11519 raft_consensus.cc:2368] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:05.489691 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:05.294234 17085 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.408735 16785 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.434369 11333 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:05.435583 16803 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:05.441294 16874 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:05.511763 11268 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 11 to 16, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 16 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:05.512666 17085 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:05.513383 12069 pending_rounds.cc:85] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 9
I20260709 13:17:05.513687 12069 pending_rounds.cc:107] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:17:05.513986 12069 raft_consensus.cc:2889] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:05.514937 16954 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.444170 16786 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:05.433178 17225 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:05.516489 17225 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.517730 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.445324 16532 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:05.520095 16532 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.520949 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:05.446033 11335 leader_election.cc:304] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:05.522099 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:05.522428 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 76477.2us
I20260709 13:17:05.446373 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:05.422950 17170 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:05.446767 17188 consensus_queue.cc:1059] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:05.449278 17189 consensus_queue.cc:260] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.424748 11366 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:17:05.443053 17210 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.487073 11361 rpcz_store.cc:276] 0709 13:17:04.332226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.332346 (+   120us) service_pool.cc:224] Handling call
0709 13:17:04.332679 (+   333us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:04.481798 (+149119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:04.481870 (+    72us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:04.488164 (+  6294us) log.cc:844] Serialized 52 byte log entry
0709 13:17:05.419725 (+931561us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:17:05.419766 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.419835 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:05.419875 (+    40us) raft_consensus.cc:1685] finished
0709 13:17:05.419885 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.419979 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82170,"raft.queue_time_us":88172,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":28588,"thread_start_us":949937,"threads_started":2,"wal-append.queue_time_us":548925}
W20260709 13:17:05.543169 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52774.5us
I20260709 13:17:05.495779 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:05.502823 17074 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:05.547860 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116f1226682347599c239ad3bee95b41" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.505247 17213 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.505571 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:05.552453 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:05.508061 16578 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:05.435086 17222 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.556296 17222 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.515620 16786 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.557410 16786 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:05.423175 16881 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.520401 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:05.427219 17187 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:05.439293 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:05.564464 12064 raft_consensus.cc:2393] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 17: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:05.564611 11516 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:05.565663 16881 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:05.566691 17075 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:05.567137 17075 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.567516 17075 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:05.567798 11511 raft_consensus.cc:1275] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:17:05.513706 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 116336us
I20260709 13:17:05.527729 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.530586 11364 raft_consensus.cc:1240] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
W20260709 13:17:05.544549 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 110229us
I20260709 13:17:05.435549 11439 rpcz_store.cc:276] 0709 13:17:03.967068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.967309 (+   241us) service_pool.cc:224] Handling call
0709 13:17:03.967623 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.967814 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:17:04.028223 (+ 60409us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.047028 (+ 18805us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.047459 (+   431us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.209033 (+161574us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b18003db4c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:04.357788 (+148755us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b18003db4c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911557 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:05.143947 (+786159us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b18003db4c0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35bc9f6060 00007f35bca08716 00007f35bca19287 00007f35bca19208
0709 13:17:05.407992 (+264045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:05.408048 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:17:05.408097 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.408182 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.408312 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13759,"prepare.queue_time_us":412021,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":62149,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":31066,"spinlock_wait_cycles":32375168,"thread_start_us":263884,"threads_started":1}
I20260709 13:17:05.545629 11515 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:17:05.546402 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:05.572701 11333 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:05.499756 17144 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:05.573736 17144 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.575757 17144 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.579438 11516 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 17.
I20260709 13:17:05.513593 17057 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:05.590457 11268 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 11, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.588244 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a142ac56caa42a08e0a838dc87ac3b3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.588522 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.589056 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.589335 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.589986 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:05.590608 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:05.598482 12062 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:17:05.600942 11369 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:50722
I20260709 13:17:05.602027 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.602689 12067 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 18.
I20260709 13:17:05.603567 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.604315 12068 raft_consensus.cc:2393] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:05.605997 11409 leader_election.cc:400] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:05.608928 11409 leader_election.cc:403] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:05.539819 17215 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.583984 17075 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.586228 17168 raft_consensus.cc:2749] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:05.621711 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:05.625283 11268 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 13, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.630123 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:05.515664 16756 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:17:05.592567 17158 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:05.546407 16785 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:05.637975 11411 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.638232 17190 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:05.630396 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:05.649273 17075 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.650359 17213 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:17:05.595651 16838 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:05.592960 17074 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:05.670238 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 96345.3us
I20260709 13:17:05.661350 11268 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 11, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 11 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:05.634567 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:05.632413 17207 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:17:05.674651 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44337us
I20260709 13:17:05.679780 11369 raft_consensus.cc:606] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:05.654906 17189 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:05.686699 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 255653us
W20260709 13:17:05.688225 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 53730.6us
I20260709 13:17:05.657250 17222 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:05.712692 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:05.718685 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:05.657145 17158 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.628108 17215 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.589164 16786 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.684870 17085 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:05.691478 17225 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:05.713907 11268 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 12 to 14, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 14 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.714758 17190 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:05.735356 17226 raft_consensus.cc:3055] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:05.744534 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:05.744743 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.749333 11334 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:05.748492 12067 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:05.719111 17187 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:05.751236 11438 raft_consensus.cc:1240] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:05.759923 17205 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.760598 17205 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.770280 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.771004 11442 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
W20260709 13:17:05.689643 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 183553us
I20260709 13:17:05.802627 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.691704 17207 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.780294 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:05.769325 11268 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 14, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 14 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.716584 17222 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.813901 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.815780 11442 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
W20260709 13:17:05.809015 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 122974us
I20260709 13:17:05.810890 11268 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:17:05.816392 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.686996 16838 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.813490 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:05.753207 17187 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.826010 11334 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:05.826552 11268 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.820210 12033 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:05.827675 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 283330us
I20260709 13:17:05.754026 17085 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:17:05.847914 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 242032us
I20260709 13:17:05.836396 12034 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:05.752697 17226 raft_consensus.cc:740] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:05.803926 16756 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:05.849432 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.812318 17190 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:05.829701 17187 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:05.854759 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:05.851529 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 111071us
I20260709 13:17:05.826205 17255 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:05.885279 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:05.899127 11442 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:05.887084 11268 catalog_manager.cc:5697] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.855235 17254 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:05.909463 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:05.927196 11441 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:17:05.910995 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:05.863322 17253 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:05.912932 17057 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:05.909159 17008 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:05.863077 17186 consensus_queue.cc:1059] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:17:05.883695 17276 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:05.949440 11268 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 4 to 8, term changed from 8 to 9, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68), VOTER 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) evicted. New cstate: current_term: 9 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.863464 16756 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:05.945885 17191 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.915516 17272 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:05.862120 17085 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.906039 17255 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.963637 17057 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:05.984366 11438 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:17:05.976500 16785 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.972251 17274 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:05.974632 17074 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:05.993525 11268 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.997226 11268 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.987489 17057 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:05.985460 17215 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:06.007290 17267 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:17:06.013073 17146 raft_consensus.cc:493] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:05.986101 17205 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.023674 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:05.951839 17226 consensus_queue.cc:260] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:06.026676 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 200749us
W20260709 13:17:06.027801 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 173711us
I20260709 13:17:06.027896 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.028817 11484 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:06.029520 11334 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.029888 11484 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:06.030761 16838 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.104935 17039 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:06.105110 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 75285.1us
I20260709 13:17:06.030964 16874 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:05.983606 17191 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:05.998915 17158 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.992007 17274 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:06.107781 17294 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:06.051017 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.108417 17294 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:06.109591 17274 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.110389 11443 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:06.110628 17294 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.112869 17294 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:06.035437 17270 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.225s	user 0.001s	sys 0.001s
I20260709 13:17:06.036473 16756 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.036981 17296 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:06.040979 11334 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.004113 16786 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:06.025274 16794 consensus_meta.cc:287] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.001s
I20260709 13:17:06.044430 17255 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:05.983691 17190 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.045707 17292 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:17:06.069419 17201 raft_consensus.cc:3055] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:06.047183 17303 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:06.049599 17234 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:17:06.050707 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:06.049014 17169 consensus_meta.cc:287] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.000s
I20260709 13:17:06.051689 17226 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:06.053217 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 205741us
I20260709 13:17:06.058862 17207 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.136276 16794 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:06.080646 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:06.083582 17222 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.085224 17080 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:06.089082 17213 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
I20260709 13:17:06.095618 11268 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.098321 17215 raft_consensus.cc:493] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:06.164712 17080 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.139569 17057 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.101572 17299 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:06.166145 17299 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.103324 17146 raft_consensus.cc:515] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.115115 11408 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:06.116168 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 193629us
W20260709 13:17:06.170964 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 55925.2us
I20260709 13:17:06.171496 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114b7c3bbf65457b98b17b61f3c64839" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:06.116691 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 432752us
I20260709 13:17:06.172487 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.119567 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:06.173169 11443 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:06.175055 11516 raft_consensus.cc:1275] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:17:06.175433 11516 pending_rounds.cc:85] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 8
I20260709 13:17:06.175668 11516 pending_rounds.cc:107] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:17:06.175881 11516 raft_consensus.cc:2889] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:06.177285 11333 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.177783 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:06.123107 11442 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
I20260709 13:17:06.178675 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.180985 11408 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.128463 12066 raft_consensus.cc:1275] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:17:06.113396 17294 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:06.133620 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 110003us
W20260709 13:17:06.067245 16774 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Time spent flushing consensus metadata: real 0.613s	user 0.007s	sys 0.000s
I20260709 13:17:06.183204 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:06.183754 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:06.184208 11408 leader_election.cc:400] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 15. Current term is 16.
W20260709 13:17:06.137828 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 96932.9us
I20260709 13:17:06.185019 11408 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.188998 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.192701 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab171deb5e946c8b074a4c091542a78" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.192709 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.194312 11519 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
W20260709 13:17:06.140290 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.107808 17191 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:06.065696 17265 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
I20260709 13:17:06.157619 12034 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:06.163610 17304 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:06.140753 17296 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:06.165355 17215 raft_consensus.cc:515] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.137483 17201 raft_consensus.cc:740] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:06.192817 17055 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:06.215080 17201 consensus_queue.cc:260] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:06.215024 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.065775 17187 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.216495 11363 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:06.195294 17293 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:06.218181 17187 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.165725 11268 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 11 to 14, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 14 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:06.219482 11268 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.194653 11514 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:06.188468 11409 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.198472 16954 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:17:06.216424 17201 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:06.221014 17255 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:06.222059 17055 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:06.221813 11268 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 12 to 13, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.198570 16756 consensus_queue.cc:1048] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:17:06.224997 17146 leader_election.cc:290] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.224715 11268 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.198051 17274 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:06.195477 17169 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:06.226251 11268 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 12 to 15, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 15 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:06.227537 17298 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.228178 17298 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.230785 17298 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.222482 17055 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:06.206651 17188 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.232038 17188 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.233043 17255 raft_consensus.cc:2749] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:06.234450 17188 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.237469 12033 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:06.238829 17276 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:06.234102 11268 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.239459 17276 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:06.226195 17274 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:06.239987 12066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:06.240710 12066 raft_consensus.cc:606] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:06.240087 11268 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:06.241991 11268 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 10 to 12, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 12 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:06.216670 17215 leader_election.cc:290] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.194031 17085 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.197039 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:06.197507 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:06.167909 17218 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:06.187037 11335 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:06.201370 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.203584 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.203750 11408 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:06.179931 17039 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.209215 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.187570 17207 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:06.228348 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 44002.6us
I20260709 13:17:06.314411 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:17:06.243721 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 86194.9us
I20260709 13:17:06.314803 11439 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:06.196421 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.315083 11512 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:06.220110 17294 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.239996 17274 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:06.316695 17201 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:06.317250 17201 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.317696 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.231001 17080 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.318915 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:06.319533 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.319024 17207 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:06.321277 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:06.319126 16532 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:06.214361 17301 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:06.285924 17254 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:06.287318 17057 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.288136 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.244282 17148 consensus_queue.cc:1059] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:17:06.185973 17213 raft_consensus.cc:2749] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:17:06.239692 16811 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:06.289633 11513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:17:06.291937 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 88280.3us
I20260709 13:17:06.369977 11513 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:06.292497 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 105527us
I20260709 13:17:06.324340 17215 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:17:06.137036 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 246354us
I20260709 13:17:06.335098 17201 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.384604 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.385434 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dab116931a64024ac61a35a6d29a836" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.325047 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:06.321970 11518 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:17:06.343080 17299 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.343530 17294 consensus_queue.cc:1048] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:17:06.349975 11517 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:17:06.240449 17276 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.322813 17039 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.408577 17249 consensus_queue.cc:1048] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:17:06.350760 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.430322 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.431501 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.432535 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.433115 11516 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 18.
I20260709 13:17:06.369819 16811 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.460320 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.360853 12033 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:06.461344 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 100585us
I20260709 13:17:06.462086 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:06.461508 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.464151 17254 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:06.464811 17254 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:06.464975 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.465943 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98fc3c2677604f3bb787b67c5e5b6012" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.466077 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.467139 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.467332 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.334343 17301 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:06.470198 17301 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:06.470911 11361 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 17: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:06.468209 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.364766 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.475871 12066 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:06.477149 17055 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:06.362097 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:06.479009 11440 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:06.460623 16811 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:06.365464 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:06.366087 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.367056 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.368108 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.368685 12034 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:06.374586 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.374737 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.322644 17074 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:06.321923 17207 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.382150 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.334741 16532 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:06.384343 11408 leader_election.cc:304] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:06.283717 11268 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 13 to 14, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 14 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.502240 11443 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:06.347913 17080 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:06.402139 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887116b7f3e447028e317f03386bf8d0" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.352191 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:06.511492 11515 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 13: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:06.438938 11368 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:17:06.415999 17278 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:06.417155 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.441725 11510 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:17:06.425966 17039 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.448563 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.448895 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.451922 16881 raft_consensus.cc:3055] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:06.453027 17201 consensus_queue.cc:1059] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:06.453976 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:06.458745 11513 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:06.459758 17218 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:06.519178 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:06.468422 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.468827 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:06.520236 11513 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:06.474928 17274 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.522936 16881 raft_consensus.cc:740] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:06.523669 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:06.523907 16881 consensus_queue.cc:260] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.524996 16881 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:17:06.520551 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 51833us
I20260709 13:17:06.529198 11409 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:06.518297 11362 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
W20260709 13:17:06.484174 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 115553us
I20260709 13:17:06.488878 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:06.495736 11441 raft_consensus.cc:1240] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:06.501245 11518 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 11.
W20260709 13:17:06.505021 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 120748us
I20260709 13:17:06.338030 17085 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.505733 12066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:06.563086 12066 raft_consensus.cc:606] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:06.563628 12066 raft_consensus.cc:614] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:06.512435 11510 pending_rounds.cc:85] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Aborting all ops after (but not including) 12
I20260709 13:17:06.564229 11510 pending_rounds.cc:107] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:17:06.564508 11510 raft_consensus.cc:2889] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:06.512655 11517 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:06.503135 17057 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:06.375068 17144 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:06.527278 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 73375.9us
I20260709 13:17:06.569101 11408 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:06.361037 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.571038 11514 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:06.572732 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.479382 17055 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.574420 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:06.533316 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.578838 11409 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.580319 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.576061 11408 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:06.585389 11409 leader_election.cc:304] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:06.587268 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 56025.3us
I20260709 13:17:06.589432 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.523029 17308 raft_consensus.cc:3055] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:06.531504 17255 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:06.602415 11335 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:06.524646 17305 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.527841 17207 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:06.515751 17249 raft_consensus.cc:2749] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:06.601317 12034 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:06.596750 17213 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:06.596155 12063 raft_consensus.cc:2410] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:17:06.566895 17057 raft_consensus.cc:493] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.607688 11333 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:06.587061 17294 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:06.507315 17074 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.594539 16756 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:06.596398 17308 raft_consensus.cc:740] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:06.572196 17222 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:17:06.593971 17299 raft_consensus.cc:2749] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:06.527593 17080 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.605048 17213 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.628345 11437 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 18. Current term is 19. Ops: []
I20260709 13:17:06.602545 17158 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:06.618503 17055 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:06.584329 17144 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.563143 17278 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:06.631412 11252 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7a5787304dfe4697baeaeb367ee7dda3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 34a42aa6aa0a4778a532c6433d70b981 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:06.609747 17057 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:06.670270 11252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 50951.9us
I20260709 13:17:06.676321 11334 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.634179 17213 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:06.652479 16794 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:17:06.611613 17305 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.693136 17301 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:06.669936 17298 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:06.696645 17074 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.619551 16532 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:06.699326 11440 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:06.704025 16954 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:06.652551 17226 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:06.701973 17278 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.721334 17169 raft_consensus.cc:2764] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:17:06.723479 17298 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:06.721946 17226 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:06.742373 16811 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.743561 12069 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 18: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:06.739182 17226 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:06.751926 16811 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:06.754031 17226 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.768852 11512 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:06.716065 17308 consensus_queue.cc:260] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:06.775480 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 187733us
W20260709 13:17:06.795130 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 270724us
I20260709 13:17:06.738848 17278 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.814082 11515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:06.819439 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:17:06.842068 11483 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.815898 17274 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:06.917071 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 323347us
I20260709 13:17:06.840864 17308 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:06.947002 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 100751us
I20260709 13:17:06.952026 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:06.956189 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:06.953215 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99667fcf16014dc1b9b174f0a37d16af" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:06.946795 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:06.860241 17300 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:06.974884 17300 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.977051 17300 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:06.978561 17300 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.960160 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:06.897854 17085 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:06.960315 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 370926us
I20260709 13:17:06.818842 17278 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:06.946427 17144 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.973762 17210 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.966626 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:06.962101 17080 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:06.995303 11362 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 18: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:06.997521 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.979012 17300 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.003731 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.990656 17226 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.010851 11409 leader_election.cc:304] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:06.846227 17298 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.002693 16881 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:06.997771 17210 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.042227 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:07.052299 11510 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:07.043962 17210 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:07.028587 17146 raft_consensus.cc:2749] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:07.048524 17305 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.054060 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:07.071353 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:07.058058 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:07.100001 11436 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:07.090039 11367 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:07.011813 17144 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:07.013586 17169 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:07.128192 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 225892us
I20260709 13:17:07.129076 12034 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:07.041395 17278 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.018224 17300 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:07.095577 17292 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:07.089919 17301 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:07.157400 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.090124 17207 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:07.079916 17201 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:07.158663 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:07.161298 11368 raft_consensus.cc:1240] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:07.102712 17146 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:07.203725 17146 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:07.206349 17146 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:07.152804 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.069860 17226 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:07.207674 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:07.207957 17226 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.210328 17226 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.165546 11437 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:07.035898 16881 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:07.161439 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:07.016673 16874 consensus_queue.cc:1059] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.125s
I20260709 13:17:07.201152 11444 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:17:07.217442 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:07.212158 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:07.219214 11445 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
W20260709 13:17:07.225378 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 64033.8us
I20260709 13:17:07.145290 17210 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:07.088356 17057 raft_consensus.cc:515] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.229269 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.232569 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.167822 17272 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.155855 17292 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:07.249117 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:07.256183 12060 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:07.235692 12066 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:07.258894 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.250397 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.259824 12067 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:17:07.252830 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.253499 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "572d79fba792428db7fcc1994fff81f7" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.263471 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:07.331811 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 68447.5us
I20260709 13:17:07.333168 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:07.334484 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 74439.3us
W20260709 13:17:07.391912 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 139432us
I20260709 13:17:07.260429 17292 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:07.450044 16874 consensus_queue.cc:1059] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:07.451000 17334 raft_consensus.cc:3055] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:07.427946 16838 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:07.428900 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:07.430962 17057 leader_election.cc:290] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:07.432749 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:07.480940 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 52127.9us
I20260709 13:17:07.439775 17300 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:07.266422 17210 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:07.460036 16881 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:07.236860 17323 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:07.416755 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:07.478997 12060 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 19.
I20260709 13:17:07.481719 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:07.440644 17213 consensus_meta.cc:287] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.000s
W20260709 13:17:07.485975 12035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.010s	sys 0.024s
I20260709 13:17:07.262256 17207 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.504717 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:07.508961 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 92293.9us
I20260709 13:17:07.487540 17300 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.264421 17288 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.224s
I20260709 13:17:07.415704 17169 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:07.518541 11485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.008s	sys 0.027s
W20260709 13:17:07.565120 11485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.008s	sys 0.027s
W20260709 13:17:07.564945 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 83307us
I20260709 13:17:07.558614 17213 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:07.481092 17334 raft_consensus.cc:740] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:07.546835 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.547559 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.496536 17269 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:07.551667 12063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54370 (request call id 1461) took 1174 ms. Trace:
I20260709 13:17:07.267603 17272 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.519052 17274 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:07.555830 12066 raft_consensus.cc:1240] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 18. Current term is 19. Ops: []
I20260709 13:17:07.557443 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.573556 11515 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:07.607316 11368 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
W20260709 13:17:07.555385 12035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.010s	sys 0.024s
I20260709 13:17:07.605767 11409 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:07.480646 17170 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:07.573971 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:07.609941 11517 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:17:07.611210 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:07.572468 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:07.601897 12063 rpcz_store.cc:276] 0709 13:17:06.367719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.468001 (+100282us) service_pool.cc:224] Handling call
0709 13:17:07.541995 (+1073994us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540039fdd4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bca022fa 00007f35c19c3b17 00007f35baf3e6d4 00007f35baf3e662 00007f35baf3e591 00007f35baf3e50c 00007f35baf3d7c2 00007f35bad3948c 00007f35bad388c6 00007f35bad3826f
0709 13:17:07.542157 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278097792}
I20260709 13:17:07.572752 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:07.614959 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:07.620142 11514 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:07.619987 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:07.569752 17287 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:07.625648 11367 raft_consensus.cc:2410] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:17:07.619035 17148 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:17:07.626688 17330 log.cc:927] Time spent T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.367s	user 0.000s	sys 0.000s
I20260709 13:17:07.626931 17288 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:07.628196 12061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54370 (request call id 1460) took 1260 ms. Trace:
I20260709 13:17:07.628568 12061 rpcz_store.cc:276] 0709 13:17:06.367434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.367539 (+   105us) service_pool.cc:224] Handling call
0709 13:17:06.387358 (+ 19819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:06.406350 (+ 18992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:06.406379 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:06.513292 (+106913us) log.cc:844] Serialized 26 byte log entry
0709 13:17:07.000713 (+487421us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:07.000763 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.000838 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:07.627979 (+627141us) raft_consensus.cc:1685] finished
0709 13:17:07.627998 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.628119 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10578,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":81,"thread_start_us":383301,"threads_started":1,"wal-append.queue_time_us":569262}
W20260709 13:17:07.642699 17327 log.cc:927] Time spent T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:17:07.607585 16881 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.646064 16881 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.648257 16881 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:07.621580 11482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.019s	sys 0.027s
W20260709 13:17:07.650547 11482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.019s	sys 0.027s
I20260709 13:17:07.602316 17305 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.606770 17274 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.565938 17300 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:07.626215 17331 log.cc:927] Time spent T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
W20260709 13:17:07.652540 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46859.4us
I20260709 13:17:07.677074 11409 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:07.622179 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:07.655019 11366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2782) took 1274 ms. Trace:
I20260709 13:17:07.622319 11333 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
W20260709 13:17:07.679987 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 57764us
I20260709 13:17:07.662233 11364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37086 (request call id 2784) took 1281 ms. Trace:
I20260709 13:17:07.611671 17170 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:17:07.651475 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.674474 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:07.680915 11364 rpcz_store.cc:276] 0709 13:17:06.380923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.382020 (+  1097us) service_pool.cc:224] Handling call
0709 13:17:06.382261 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:06.382443 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:17:06.763076 (+380633us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.661818 (+898742us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:07.661910 (+    92us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:07.661994 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.662060 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.662174 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97934,"prepare.queue_time_us":58704,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":87054,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":90,"thread_start_us":1180618,"threads_started":3,"wal-append.queue_time_us":470562}
I20260709 13:17:07.570498 17169 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.673807 17274 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:07.657130 17326 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:07.678059 11366 rpcz_store.cc:276] 0709 13:17:06.380409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.381096 (+   687us) service_pool.cc:224] Handling call
0709 13:17:06.381481 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:06.650511 (+269030us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:17:06.686734 (+ 36223us) log.cc:844] Serialized 14 byte log entry
0709 13:17:06.989636 (+302902us) log.cc:844] Serialized 14 byte log entry
0709 13:17:06.989847 (+   211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:06.989888 (+    41us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:17:06.989941 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.446954 (+457013us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b1800353ec0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:07.489970 (+ 43016us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b1800353ec0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419
0709 13:17:07.642153 (+152183us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b4400022b4c. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b9837c85 00007f35b98386a2 00007f35bc5aa1db 00007f35bc5a5aaf 00007f35bc59986e 00007f35bc598bb6 00007f35bc5987d8 00007f35bc5a2221 00007f35bc5a21d9 00007f35bc5a2169
0709 13:17:07.646262 (+  4109us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b1800353ec0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:07.648556 (+  2294us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b1800353ec0. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:07.654749 (+  6193us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54002dad54. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:07.654808 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.654951 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36116,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":153,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":188,"spinlock_wait_cycles":737148160,"thread_start_us":302034,"threads_started":1,"wal-append.queue_time_us":729156}
W20260709 13:17:07.674383 17343 log.cc:927] Time spent T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:17:07.698575 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.673503 17305 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.601435 17210 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.712301 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:07.690541 17146 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:07.689929 17190 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:07.720590 17190 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.721031 17190 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:07.721964 17080 raft_consensus.cc:3055] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:17:07.722420 17080 raft_consensus.cc:740] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:07.679837 17192 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:17:07.684440 17170 raft_consensus.cc:3060] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:07.688858 17218 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:07.692323 16578 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.730543 16578 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.729993 17218 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:07.733076 17218 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:07.754926 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 419493us
I20260709 13:17:07.760723 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:07.770489 12062 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.002s	sys 0.022s
I20260709 13:17:07.690078 17336 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:17:07.767436 17317 log.cc:927] Time spent T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:07.751925 17207 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:07.777180 17308 consensus_meta.cc:287] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.825s	user 0.007s	sys 0.003s
I20260709 13:17:07.753636 17146 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 19 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:07.810964 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:17:07.812729 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 478662us
I20260709 13:17:07.723191 17080 consensus_queue.cc:260] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.843812 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:07.844449 11436 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:17:07.814988 12062 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.003s	sys 0.022s
I20260709 13:17:07.828490 11511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36660 (request call id 2763) took 1613 ms. Trace:
I20260709 13:17:07.828768 17305 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.829757 17169 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.830739 17336 raft_consensus.cc:3055] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:07.696007 17334 consensus_queue.cc:260] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:07.813499 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52878.1us
I20260709 13:17:07.860044 11334 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:07.813356 17215 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.205s	user 0.009s	sys 0.028s
W20260709 13:17:07.873586 17215 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.268s	user 0.010s	sys 0.028s
I20260709 13:17:07.876435 11367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.877857 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce024e6afd54cf0992c0af40df6a4eb" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:07.878551 11365 raft_consensus.cc:3060] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:07.881326 11333 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:07.882164 11370 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 15. Current term is 16. Ops: []
W20260709 13:17:07.849262 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 234354us
I20260709 13:17:07.858965 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.825387 17213 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:07.861886 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.830915 16578 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.848273 11511 rpcz_store.cc:276] 0709 13:17:06.211144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.323705 (+112561us) service_pool.cc:224] Handling call
0709 13:17:06.363290 (+ 39585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:06.673753 (+310463us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:17:06.998631 (+324878us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.504752 (+506121us) log.cc:844] Serialized 14 byte log entry
0709 13:17:07.504909 (+   157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:07.504968 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:17:07.505014 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:07.644104 (+139090us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b1800296b60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98df2a6 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1
0709 13:17:07.645462 (+  1358us) spinlock_profiling.cc:243] Waited 8.65 ms on lock 0x7b1800296b60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:07.822610 (+177148us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54000ea954. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9fee01 00007f35bc9fd83e 00007f35c19c2419 00007f35baf39bd4 00007f35baf39b62 00007f35baf39a91 00007f35baf39a0c 00007f35baf38cc2 00007f35bad3948c 00007f35bad388c6
0709 13:17:07.822773 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.824031 (+  1258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12674,"prepare.queue_time_us":539058,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":21722,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":25939,"spinlock_wait_cycles":766168960,"thread_start_us":793259,"threads_started":2,"wal-append.queue_time_us":727753}
I20260709 13:17:07.907254 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.869288 11445 raft_consensus.cc:1240] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 18. Current term is 19. Ops: []
I20260709 13:17:07.873605 11514 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:07.841318 16811 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:07.914577 11366 raft_consensus.cc:2393] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:07.918678 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:07.919449 11368 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:17:07.927821 11333 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:07.864543 17336 raft_consensus.cc:740] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:07.938488 17254 consensus_queue.cc:1059] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:07.940002 17272 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:07.909988 17188 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:17:07.939273 17254 consensus_queue.cc:1243] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer 01ae50c877d04ba687e4e06259ba5581 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:17:07.927786 16954 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:07.950199 11365 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:07.956007 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.930617 17342 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.956691 12063 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:07.958194 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:07.962572 11511 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:07.959189 17201 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:07.963842 17341 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:17:07.934386 17338 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:07.961006 11519 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:07.967793 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.967351 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:07.952069 16532 raft_consensus.cc:493] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.967204 16794 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:17:07.971472 17336 consensus_queue.cc:260] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.973320 11483 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:07.950555 17272 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:07.980042 11483 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:07.973920 17313 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 1.113s	user 0.005s	sys 0.009s
I20260709 13:17:07.983588 11409 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:07.986187 11409 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:07.966410 17342 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.963850 17201 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.008420 17190 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.009047 17201 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:07.981518 17254 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:08.009837 17254 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.010208 17300 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.011023 17190 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:07.989513 17189 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:07.986500 17298 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:07.984630 17339 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:08.016391 17339 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.018581 17339 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:07.961015 17146 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.019613 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:07.987186 17305 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:08.020431 17298 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.978039 16874 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:07.978297 16532 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:08.003756 17292 consensus_meta.cc:287] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Time spent flushing consensus metadata: real 0.563s	user 0.004s	sys 0.007s
I20260709 13:17:08.022683 17292 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.023594 17189 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.024802 17292 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.006079 11436 raft_consensus.cc:2468] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:08.012912 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:07.972564 17338 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:07.989740 17326 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:07.979717 16954 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:17:08.037325 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.014298 17169 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:08.023747 16532 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.044858 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.022503 17298 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.039053 17292 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:08.042044 17274 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.054342 17274 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.055979 17169 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.056522 17169 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.056666 17170 raft_consensus.cc:515] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.058534 17169 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.048341 17189 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.069140 16954 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.069728 16954 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.045898 17326 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:08.071771 16954 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.072551 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.077751 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.078373 11366 raft_consensus.cc:3060] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:08.086988 11365 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:08.089499 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:08.108618 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.109341 11365 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:08.112032 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.112995 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:08.113783 12062 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 13: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:08.114318 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.114928 11512 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:08.117974 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:08.121341 17201 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.121809 17201 raft_consensus.cc:493] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.122167 17201 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.130519 17201 raft_consensus.cc:515] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.133929 17201 leader_election.cc:290] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:08.139572 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.140283 11365 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.150018 11365 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:08.154419 11333 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:08.151330 11483 leader_election.cc:304] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:08.118938 17278 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:08.157990 17301 raft_consensus.cc:2764] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:17:08.160211 17148 raft_consensus.cc:2804] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:08.190538 17254 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.159881 17342 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:08.203056 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.163611 17278 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:08.209148 12060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.209649 17278 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:08.212222 17369 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.132s	user 0.000s	sys 0.003s
I20260709 13:17:08.212759 12064 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:08.215492 12067 raft_consensus.cc:1240] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:08.216526 17148 raft_consensus.cc:697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
W20260709 13:17:08.217286 11484 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.219645 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:08.219664 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.220443 12062 raft_consensus.cc:2393] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 18: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:08.221765 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:08.226145 16578 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.226815 16578 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.217442 17148 consensus_queue.cc:237] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.228147 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.228734 12067 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:08.228906 16578 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.234364 17039 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:08.234972 17039 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:08.235868 17039 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.236869 17039 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:08.237335 17338 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:08.238427 17287 consensus_meta.cc:287] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: Time spent flushing consensus metadata: real 0.587s	user 0.006s	sys 0.001s
I20260709 13:17:08.238934 17287 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:08.239530 11516 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:08.243696 17170 leader_election.cc:290] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:08.246292 17334 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:08.238590 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.247691 12062 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:17:08.248113 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.248749 11511 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:17:08.249137 17348 raft_consensus.cc:993] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:08.249434 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:08.249588 17348 raft_consensus.cc:1081] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:08.247741 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:08.251302 16578 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:08.251688 16578 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:08.252022 16578 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:08.255928 17293 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.256400 17292 raft_consensus.cc:493] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.256567 17207 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.257032 17207 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.259157 17207 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.260293 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.260898 11443 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:08.261875 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.262451 11436 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:17:08.256431 17293 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.264150 11512 raft_consensus.cc:1240] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 15. Current term is 16. Ops: [14.13-15.14]
I20260709 13:17:08.264081 17080 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:08.265337 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.266546 17292 raft_consensus.cc:515] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.268468 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.263629 11366 raft_consensus.cc:2468] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:08.269948 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.270566 11436 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:08.274134 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:08.274933 11518 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 16: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:08.269222 11361 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:08.275393 11516 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 18.
I20260709 13:17:08.279874 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:08.279995 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.273636 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:08.280488 11443 raft_consensus.cc:3060] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.281872 12033 leader_election.cc:304] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:08.277863 11409 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.279240 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.282943 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.282631 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.284520 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.284983 11366 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:08.285945 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.286034 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.286788 12068 raft_consensus.cc:2410] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:17:08.288072 17344 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.289613 17344 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.289994 17057 raft_consensus.cc:2804] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:08.290556 17057 raft_consensus.cc:697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:08.290755 17336 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:08.291487 17057 consensus_queue.cc:237] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.292361 17344 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.288058 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.296437 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.297127 11518 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:08.300097 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.300686 12068 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:08.301756 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:08.305398 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.306162 11436 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:08.324867 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.360028 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:08.312546 17361 log.cc:927] Time spent T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.063s	user 0.001s	sys 0.009s
I20260709 13:17:08.385869 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fc774e6f3a42cf890bd8e9fb8f7635" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.381194 11444 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:17:08.345868 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.386247 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.403126 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.277235 17292 leader_election.cc:290] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.307431 16532 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.421983 16532 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.423341 11364 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:08.311774 17169 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:08.423857 17169 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.424243 17169 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:08.425136 17300 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:08.425963 16532 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.307682 17293 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.427445 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.427681 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.428439 11366 raft_consensus.cc:2393] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 13: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:08.428531 11362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.317042 11443 raft_consensus.cc:2468] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:08.357353 17301 consensus_queue.cc:1059] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:17:08.348763 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.318349 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.432587 11518 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:08.307765 17144 raft_consensus.cc:2804] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:08.433568 17144 raft_consensus.cc:697] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:08.295038 17308 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:08.337821 17168 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.337410 17055 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.405828 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.406733 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.434610 17144 consensus_queue.cc:237] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.356630 17287 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.412189 16578 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.420135 11442 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:08.434453 17168 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.420307 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.435940 17287 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.437805 17168 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.428555 11363 raft_consensus.cc:2393] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 10: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:08.441254 11334 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.432066 17301 raft_consensus.cc:3055] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:17:08.363338 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:08.401098 11436 raft_consensus.cc:2410] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:17:08.407363 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:08.423517 17313 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:17:08.445088 11408 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:08.442359 11439 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:08.452103 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.452226 11408 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.452464 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.439218 11510 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:08.454842 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad3ab48ae4bf4452a19c99c71fd014f1" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:08.443969 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd0c27e51bd41b5959c50a56353166e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.457732 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:08.458699 11335 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:08.434926 17055 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.460626 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.464591 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:08.489732 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:08.522379 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 159146us
I20260709 13:17:08.524230 12032 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:08.525942 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 65392.4us
I20260709 13:17:08.450699 17301 raft_consensus.cc:740] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:08.536489 11439 raft_consensus.cc:2468] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 10.
I20260709 13:17:08.456844 17080 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.460795 17326 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:08.541126 11436 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:08.466256 17288 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.554276 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.555964 17231 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:08.542909 11442 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:41514
I20260709 13:17:08.549244 17080 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.557049 11442 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:08.465988 17207 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:08.557477 11442 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:08.557533 17207 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.558030 17207 raft_consensus.cc:3060] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:08.559242 17301 consensus_queue.cc:260] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.560480 17301 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:08.539475 17269 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:08.563012 17269 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.563472 17269 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.563692 17287 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.563074 16578 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.522164 16532 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:17:08.569929 17080 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:08.458503 17308 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.460031 17008 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:08.567173 17213 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.546379 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.464488 16803 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.556890 17272 consensus_queue.cc:1059] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:08.545457 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.591188 11510 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:08.578790 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.592123 11364 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:08.456662 17146 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:17:08.594924 11333 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
W20260709 13:17:08.583036 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 58896.9us
I20260709 13:17:08.600149 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:08.587574 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.539276 17348 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:08.584313 11519 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:17:08.577822 17308 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:08.460644 17169 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.597023 11442 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.556512 17055 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.605095 11516 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 12.
I20260709 13:17:08.610080 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.616398 11368 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:08.610903 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:08.576092 17288 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:08.617394 17288 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.598048 16803 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.598140 17336 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:08.625124 12032 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:08.630885 11437 raft_consensus.cc:2368] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15.
I20260709 13:17:08.606472 17272 consensus_queue.cc:1243] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:17:08.635843 11442 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.602308 17348 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.643019 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:08.645970 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:08.598270 17213 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.649497 17324 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.649938 17324 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.650259 17324 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.651151 17213 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.652670 17213 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:08.653193 17213 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.641767 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:08.613963 16794 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.656208 17213 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.622471 17338 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:17:08.661995 11482 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.633260 16803 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.666862 11516 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 17. Current term is 18. Ops: []
I20260709 13:17:08.648586 17039 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.647691 17348 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.623044 17207 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.656230 17298 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:08.669341 17305 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:08.650185 17074 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:17:08.662842 16794 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.685027 16794 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.665164 16532 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:08.679582 17305 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.693966 16532 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.674649 17039 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.699023 17008 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:17:08.707629 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 167302us
I20260709 13:17:08.676203 17344 consensus_queue.cc:1059] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:17:08.689129 17298 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:08.701140 17305 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.707854 16532 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:08.711357 17008 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:08.711140 17039 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.724097 17298 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:08.733253 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 79033.4us
I20260709 13:17:08.736254 11408 leader_election.cc:304] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:08.737306 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.740811 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:08.740758 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.748942 11519 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:08.751827 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.755678 17057 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:08.749193 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:08.757113 17300 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:08.757547 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:08.756296 17057 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.758315 17300 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:08.766085 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.756188 17305 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.761297 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.761616 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:08.769753 12062 raft_consensus.cc:3055] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:08.770061 12062 raft_consensus.cc:740] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:08.772383 11408 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:08.772490 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 129745us
I20260709 13:17:08.763163 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.764962 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "063871012da74857a239cdf01c76e5ab" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:08.765126 11519 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:08.765254 11364 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:08.774570 12068 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:08.779304 17213 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:08.778631 12062 consensus_queue.cc:260] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.790584 12062 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:08.786240 17213 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.791821 17213 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.792126 17274 raft_consensus.cc:493] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:08.792649 17274 raft_consensus.cc:515] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.779346 17300 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:17:08.796857 17057 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:08.797541 17008 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.799942 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.800552 17213 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.799861 17305 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:08.800308 17274 leader_election.cc:290] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.805223 12068 raft_consensus.cc:2468] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 18.
I20260709 13:17:08.806550 17213 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:08.808485 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 253955us
I20260709 13:17:08.812032 11334 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.814469 17222 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:17:08.819629 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 46503.1us
I20260709 13:17:08.820333 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:08.821558 17170 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:08.822191 17170 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:08.823141 12034 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:08.823972 17323 raft_consensus.cc:2804] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:08.824430 17323 raft_consensus.cc:697] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:08.825213 17323 consensus_queue.cc:237] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.825829 11334 leader_election.cc:304] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
W20260709 13:17:08.829497 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 46743.8us
I20260709 13:17:08.832324 17057 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:08.833257 16774 raft_consensus.cc:2749] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:08.835640 11483 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:08.837918 17305 raft_consensus.cc:2804] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:08.838366 17305 raft_consensus.cc:697] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:08.838304 11334 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:08.842721 16774 raft_consensus.cc:2749] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:08.839234 17305 consensus_queue.cc:237] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.851152 17337 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:08.852497 12034 leader_election.cc:400] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15.
I20260709 13:17:08.753207 17326 raft_consensus.cc:2749] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:08.856709 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ecdb2d3e8745889c0343749502c9f5" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:08.860307 11442 raft_consensus.cc:1240] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:08.868988 11439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:08.863986 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c1dd88ce9a04ff59ee55a115686f148" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:08.873937 11365 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:17:08.876073 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.879153 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.879745 11364 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.881402 11442 raft_consensus.cc:1240] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:17:08.882103 12066 raft_consensus.cc:1240] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:17:08.885655 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:08.886333 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:08.888388 11511 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Rejecting Update request from peer 6e87b213a4554d94825aceec352fc094 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:08.888494 12064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:08.889941 12065 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:17:08.890296 12065 pending_rounds.cc:85] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 13
I20260709 13:17:08.890498 12065 pending_rounds.cc:107] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:17:08.890722 12065 raft_consensus.cc:2889] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:08.788331 17169 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.904526 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:08.914376 17324 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.915912 17383 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:08.916433 17383 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.917266 17269 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.918288 11512 raft_consensus.cc:1240] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:08.918458 17383 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.919736 11510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:08.920644 17326 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:08.921511 17274 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:08.921878 17274 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:08.922695 17274 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:08.924016 17274 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.928726 17269 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.936620 17207 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.940636 16794 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.943054 16794 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.908891 17192 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:17:08.950083 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:08.950491 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:08.952948 17364 raft_consensus.cc:3055] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:17:08.976924 17364 raft_consensus.cc:740] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:08.953570 17254 consensus_queue.cc:1059] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:08.977767 17254 raft_consensus.cc:3055] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:08.978143 17254 raft_consensus.cc:740] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:08.977890 17364 consensus_queue.cc:260] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.954744 17340 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:08.978922 17254 consensus_queue.cc:260] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.980090 17254 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:08.960469 17288 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.982059 17288 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:08.982697 11511 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17. Ops: []
I20260709 13:17:08.958899 17008 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:08.963941 17348 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:08.979185 17364 raft_consensus.cc:3060] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:08.989410 17324 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.990077 17324 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:08.991643 17384 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:08.992055 17384 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:08.992550 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:08.977090 11365 raft_consensus.cc:2368] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:17:08.993014 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:08.994931 12033 leader_election.cc:400] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:17:08.995244 17348 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:08.995752 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:08.963757 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:17:08.997197 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.997221 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:08.995277 12033 leader_election.cc:403] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:08.999470 17348 raft_consensus.cc:3060] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:17:08.999670 11517 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:09.004032 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:09.006028 11442 raft_consensus.cc:1275] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:17:09.008330 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:09.009642 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
W20260709 13:17:09.008093 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 103660us
I20260709 13:17:09.012804 11511 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:08.934836 17308 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.024138 12062 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:09.031108 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:09.024125 11484 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:09.015547 17305 consensus_queue.cc:1048] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:17:09.031941 11436 raft_consensus.cc:2368] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15.
I20260709 13:17:09.007341 11517 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:09.035048 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:09.035714 11436 raft_consensus.cc:3060] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:09.038162 17278 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:08.992914 17384 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.046448 17384 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:09.048110 11436 raft_consensus.cc:2468] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:09.049705 12034 leader_election.cc:400] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15.
I20260709 13:17:09.050346 12034 leader_election.cc:403] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:17:09.031697 11442 raft_consensus.cc:1275] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:09.044373 16532 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:09.012805 16803 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.053897 16532 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:09.063802 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:09.076922 11511 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:09.076138 17348 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857) with higher term. Message: Invalid argument: T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:17:09.080638 11518 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:09.065026 11365 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 17: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:09.064606 12068 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:09.086462 17170 consensus_queue.cc:1059] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:09.093116 17325 raft_consensus.cc:3055] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:09.093571 17325 raft_consensus.cc:740] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:09.094353 17325 consensus_queue.cc:260] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.009893 17192 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:09.094965 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 83940us
I20260709 13:17:09.095369 17325 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:09.115811 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:09.116588 12068 raft_consensus.cc:3060] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:09.120105 11364 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:09.071525 17039 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.126366 17039 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:09.127197 11510 raft_consensus.cc:2368] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:17:09.098838 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:09.074519 17308 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:09.128888 11518 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 17: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:09.129102 12032 leader_election.cc:400] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667) with higher term. Message: Invalid argument: T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 16. Current term is 17.
I20260709 13:17:09.067499 16803 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:09.071013 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:09.131333 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:09.132119 11436 raft_consensus.cc:2393] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 17: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
W20260709 13:17:09.135907 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 86344.3us
I20260709 13:17:09.136565 12034 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:09.137964 11512 raft_consensus.cc:2468] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:09.139671 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:09.142297 11335 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:09.051631 17288 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.133831 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:09.149904 11442 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:09.148579 11511 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 13.
I20260709 13:17:09.140882 11408 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:09.156623 17278 raft_consensus.cc:2804] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:09.157212 17278 raft_consensus.cc:697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:09.158111 17278 consensus_queue.cc:237] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:09.162973 11333 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
W20260709 13:17:09.183380 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45649.7us
W20260709 13:17:09.206583 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 72823.7us
I20260709 13:17:09.211316 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:09.212384 11364 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 17: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:09.217229 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:09.210402 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:09.227572 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:09.240918 12068 raft_consensus.cc:2468] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:09.179358 17008 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:09.246230 12065 raft_consensus.cc:2468] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:09.245036 11409 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:09.183040 16803 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:09.251345 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:09.207609 17305 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:09.247836 17213 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:09.249423 17269 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:09.257085 17207 raft_consensus.cc:2804] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:09.193399 17324 raft_consensus.cc:2804] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:09.207193 17384 consensus_queue.cc:1048] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:17:09.284425 11483 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:09.282713 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:09.288965 12064 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:09.289004 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:09.290669 12062 raft_consensus.cc:2410] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:17:09.244659 17254 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15.
I20260709 13:17:09.307480 11483 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:09.308218 12062 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:09.302997 11262 catalog_manager.cc:5697] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 17. New cstate: current_term: 17 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:09.312517 12068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:09.312999 11262 catalog_manager.cc:5697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 14, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 14 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:09.321962 12065 raft_consensus.cc:2393] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:09.321300 11262 catalog_manager.cc:5697] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 13, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 13 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:09.281639 17207 raft_consensus.cc:697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:09.282384 16803 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:09.323787 11262 catalog_manager.cc:5697] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 12 to 13, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 13 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.330749 11262 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 11, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 11 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.293491 17305 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:09.332473 11262 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 16, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 16 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.333990 11262 catalog_manager.cc:5697] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 14, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 14 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:09.335359 11262 catalog_manager.cc:5697] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.336854 11262 catalog_manager.cc:5697] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 12 to 14, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 14 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.338461 11262 catalog_manager.cc:5697] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 17, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 17 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.350596 11262 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 16, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 16 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
W20260709 13:17:09.351521 12034 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.351959 11262 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.353235 11262 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 15, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 15 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:09.354478 11262 catalog_manager.cc:5697] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 12 to 14, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 14 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.355931 11262 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 13, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 13 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.358903 11262 catalog_manager.cc:5697] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 13, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 13 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.360205 11262 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 9, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 9 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.361970 11262 catalog_manager.cc:5697] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.384819 11262 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 12, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.386049 11364 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:17:09.386444 11262 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 12 to 13, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.325749 11409 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:09.321605 17039 raft_consensus.cc:2804] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:09.312566 17269 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:09.313853 17324 raft_consensus.cc:697] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:09.328299 17334 raft_consensus.cc:2804] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:09.380735 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 857937fa61dc400a81345944f0a14511 with delete_type TABLET_DATA_TOMBSTONED (TS 34a42aa6aa0a4778a532c6433d70b981 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:48450
W20260709 13:17:09.388717 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 63062.6us
I20260709 13:17:09.390069 17146 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.323660 17144 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:17:09.395095 11364 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:17:09.394910 11262 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 10 to 12, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 12 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:17:09.405138 12035 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.405444 11262 catalog_manager.cc:5697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 11, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 11 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:09.410162 11262 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.411329 11262 catalog_manager.cc:5697] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 16 to 20, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 20 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:09.405853 11409 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:09.412817 11262 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 14 to 15, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 15 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:09.414214 11262 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:09.415602 11262 catalog_manager.cc:5697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 6 to 15. New cstate: current_term: 15 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
W20260709 13:17:09.416940 11268 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56588 (request call id 239) took 14492 ms (client timeout 15000 ms). Trace:
I20260709 13:17:09.417301 11262 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 12 to 13, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 13 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:09.406883 17274 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:17:09.404143 17288 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
W20260709 13:17:09.437788 11334 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.004s	sys 0.024s
W20260709 13:17:09.438266 11334 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.004s	sys 0.024s
I20260709 13:17:09.416569 17334 raft_consensus.cc:697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:09.443006 11437 raft_consensus.cc:1275] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:17:09.406467 17039 raft_consensus.cc:697] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:09.392911 17269 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:09.449730 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:09.451690 11365 raft_consensus.cc:3060] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:09.438735 17207 consensus_queue.cc:237] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.451881 17323 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:17:09.420599 17305 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.464953 17039 consensus_queue.cc:237] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.467757 17334 consensus_queue.cc:237] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.438560 16803 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:09.417380 11268 rpcz_store.cc:269] 0709 13:16:54.917339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.918695 (+  1356us) service_pool.cc:224] Handling call
0709 13:16:55.441545 (+522850us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b54001897d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc968090 00007f35bca06f3a 00007f35c27bd1cc 00007f35c28bb37e 00007f35c0d7c2a4 00007f35c0d7c232 00007f35c0d7c161 00007f35c0d7c0dc 00007f35c0d7b392 00007f35bad3948c 00007f35bad388c6
0709 13:17:09.409445 (+13967900us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:07.456068 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:07.456702 (+   634us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:07.456754 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:17:07.456773 (+    19us) tablet.cc:662] Decoding operations
0709 13:17:07.458483 (+  1710us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:07.458574 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:07.458600 (+    26us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:17:07.628837 (+170237us) tablet.cc:701] Row locks acquired
0709 13:17:07.628886 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:17:07.629054 (+   168us) write_op.cc:270] Start()
0709 13:17:07.629236 (+   182us) write_op.cc:276] Timestamp: P: 1783603027628988 usec, L: 0
0709 13:17:07.629260 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:17:07.629995 (+   735us) log.cc:844] Serialized 9697 byte log entry
0709 13:17:08.189256 (+559261us) op_driver.cc:464] REPLICATION: finished
0709 13:17:08.595172 (+405916us) write_op.cc:301] APPLY: starting
0709 13:17:08.601370 (+  6198us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:09.040805 (+439435us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:09.040902 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:09.164902 (+124000us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:09.165304 (+   402us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:17:09.165347 (+    43us) write_op.cc:312] APPLY: finished
0709 13:17:09.165782 (+   435us) log.cc:844] Serialized 220 byte log entry
0709 13:17:09.166620 (+   838us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:09.166824 (+   204us) write_op.cc:454] Released schema lock
0709 13:17:09.166883 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:09.167365 (+   482us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:09.167488 (+   123us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":41,"spinlock_wait_cycles":71959808,"child_traces":[["op",{"apply.queue_time_us":332800,"apply.run_cpu_time_us":30590,"apply.run_wall_time_us":619075,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108527,"num_ops":25,"prepare.queue_time_us":755391,"prepare.run_cpu_time_us":30161,"prepare.run_wall_time_us":474101,"raft.queue_time_us":190805,"raft.run_cpu_time_us":13788,"raft.run_wall_time_us":119612,"replication_time_us":559810,"thread_start_us":959730,"threads_started":4,"wal-append.queue_time_us":333644}]]}
W20260709 13:17:09.492343 17389 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:17:09.444687 17388 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:09.549034 11365 raft_consensus.cc:2468] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:09.586809 12068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b9049e00f94cc1912970768037cf24"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:09.587580 12068 raft_consensus.cc:606] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:09.587908 12068 raft_consensus.cc:614] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:09.595475 17057 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.596102 17057 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.596304 17323 consensus_queue.cc:1048] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:09.598251 17057 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:09.599694 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c8fff0a9ad4f4c9b41c65f74776fd6" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:09.600546 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:09.601778 11366 raft_consensus.cc:2393] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 12: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:09.606510 11362 raft_consensus.cc:1275] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:09.616496 11409 leader_election.cc:304] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:09.617735 17388 raft_consensus.cc:2749] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.622607 17388 consensus_queue.cc:1048] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:09.492726 17324 consensus_queue.cc:237] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:09.633497 11335 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.638509 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:09.643339 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 52033.1us
W20260709 13:17:09.648141 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 52891.7us
I20260709 13:17:09.648785 11363 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:17:09.651629 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:09.654456 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:09.655349 12069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "857937fa61dc400a81345944f0a14511"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:09.655954 12069 raft_consensus.cc:606] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:09.664397 12066 raft_consensus.cc:2393] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 17: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:09.667750 11364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:09.640473 17288 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:09.685880 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:09.694981 12061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:09.695658 12061 raft_consensus.cc:606] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:09.696007 12061 raft_consensus.cc:614] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:09.691079 17187 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:09.729318 11519 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:09.691228 17325 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:17:09.689616 17288 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.746623 11334 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:09.734489 17215 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:17:09.776319 12061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:09.861549 12061 raft_consensus.cc:606] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 LEADER]: Received request to transfer leadership to TS 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:09.749632 17189 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:09.860311 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 113791us
I20260709 13:17:09.885893 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:09.888022 17404 tablet_replica.cc:333] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:09.889647 17074 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:17:09.900162 17288 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:09.863504 16803 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:09.923995 11361 raft_consensus.cc:1275] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:09.889111 17404 raft_consensus.cc:2243] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.933013 11370 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:09.948225 12062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:09.948828 12062 raft_consensus.cc:606] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:09.869730 17345 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 13:17:09.869783 17189 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:09.950975 17074 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.000569 11436 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:17:09.990715 17074 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:10.047988 11436 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:10.020143 17404 raft_consensus.cc:2272] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:10.050370 17388 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.095696 11519 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:17:10.093266 17410 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 13:17:10.097797 17406 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 13:17:10.102459 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 81768.5us
W20260709 13:17:10.116093 17408 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:17:10.116518 17144 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.117170 17144 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:10.119879 17144 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:10.129916 17402 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:17:10.134223 17074 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:10.135759 17189 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:10.087909 17388 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:10.119925 17404 ts_tablet_manager.cc:1916] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:10.147739 11517 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:17:10.150025 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 166379us
I20260709 13:17:10.153345 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:10.154023 11366 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
W20260709 13:17:10.157114 17414 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
I20260709 13:17:10.160125 11368 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:10.163561 11368 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:17:10.164130 17416 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:17:10.167115 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 230308us
I20260709 13:17:10.173287 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.174158 11437 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:17:10.204267 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 110934us
I20260709 13:17:10.204772 11440 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:17:10.205139 11440 pending_rounds.cc:85] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Aborting all ops after (but not including) 9
I20260709 13:17:10.205355 11440 pending_rounds.cc:107] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:17:10.205557 11440 raft_consensus.cc:2889] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:10.214066 17413 log.cc:927] Time spent T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:17:10.211000 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:10.248783 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 307293us
I20260709 13:17:10.296757 11441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51474 (request call id 3048) took 1161 ms. Trace:
I20260709 13:17:10.297233 11441 rpcz_store.cc:276] 0709 13:17:09.135618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.136300 (+   682us) service_pool.cc:224] Handling call
0709 13:17:09.136621 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.137238 (+   617us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:09.364130 (+226892us) log.cc:844] Serialized 14 byte log entry
0709 13:17:09.840546 (+476416us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.841054 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.841201 (+   147us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:09.841232 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.841681 (+   449us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.216016 (+374335us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b240040c958. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b97979c4 00007f35b9795acb 00007f35b9950dc9 00007f35b9950a55 00007f35b994ffb0 00007f35bc5c6488 00007f35bc5c6be8 00007f35bc59e398 00007f35bc59976a 00007f35bc598bb6
0709 13:17:10.296444 (+ 80428us) raft_consensus.cc:1685] finished
0709 13:17:10.296562 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.296698 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":335020,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":6674,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":2642,"spinlock_wait_cycles":20082560,"thread_start_us":698448,"threads_started":2,"wal-append.queue_time_us":618268}
I20260709 13:17:10.299263 17144 consensus_queue.cc:1048] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:10.301072 11437 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:17:10.301987 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 359118us
W20260709 13:17:10.302260 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54266.2us
I20260709 13:17:10.304245 16874 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:17:10.307375 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 93877.7us
I20260709 13:17:10.312623 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:10.313369 11365 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 17.
I20260709 13:17:10.314319 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
W20260709 13:17:10.316315 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 109230us
I20260709 13:17:10.318500 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:10.320123 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:10.321987 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:10.307199 17388 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:10.308423 17395 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:17:10.329469 17192 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:17:10.327788 17324 raft_consensus.cc:993] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:10.398748 17324 raft_consensus.cc:1081] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:10.319828 17213 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
W20260709 13:17:10.404614 17403 log.cc:927] Time spent T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:17:10.406826 17411 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:10.377517 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.378290 11365 raft_consensus.cc:1275] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:17:10.378690 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.409125 11516 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:10.380071 17432 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:10.377013 17193 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:10.410208 17193 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:10.412326 17193 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:10.382220 17287 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
W20260709 13:17:10.386665 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 69698.7us
I20260709 13:17:10.413143 17287 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.415113 17287 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:10.415704 17324 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:10.416975 17404 ts_tablet_manager.cc:1929] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 8.7
I20260709 13:17:10.417500 17404 log.cc:1199] T 857937fa61dc400a81345944f0a14511 P 34a42aa6aa0a4778a532c6433d70b981: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/wals/857937fa61dc400a81345944f0a14511
I20260709 13:17:10.420506 12066 raft_consensus.cc:1275] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:17:10.388801 17420 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.422658 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:10.423645 12062 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:17:10.431248 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 119365us
I20260709 13:17:10.423062 17420 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.432847 12066 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:10.433823 17420 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:10.439615 12066 raft_consensus.cc:1275] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:10.441089 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab2785abe6bd4898ac2dbb25f53e4c8b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.444499 12062 raft_consensus.cc:1275] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:17:10.450191 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 63299.2us
I20260709 13:17:10.391724 17427 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.454896 17427 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.457322 17427 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:10.446904 11519 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:17:10.463315 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:10.464159 17287 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:10.464593 17287 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:10.464944 17287 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.452203 17192 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:10.456207 17324 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.467866 17324 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.468884 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.470013 17324 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:10.473698 17287 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.474984 11517 raft_consensus.cc:2410] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:17:10.451659 17404 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48450 (request call id 2) took 1087 ms. Trace:
I20260709 13:17:10.479538 17404 rpcz_store.cc:276] 0709 13:17:09.373805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.380422 (+  6617us) service_pool.cc:224] Handling call
0709 13:17:09.887606 (+507184us) ts_tablet_manager.cc:1151] Deleting tablet 857937fa61dc400a81345944f0a14511
0709 13:17:09.887725 (+   119us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:17:10.329090 (+441365us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:10.416845 (+ 87755us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:10.451582 (+ 34737us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9406,"tablet-delete.queue_time_us":459215,"thread_start_us":297172,"threads_started":1}
I20260709 13:17:10.482234 17287 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:10.483315 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 857937fa61dc400a81345944f0a14511 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:17:10.484895 17288 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:10.487197 17288 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:10.494386 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.495100 11517 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:10.497020 12065 raft_consensus.cc:1275] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:17:10.391669 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 81591.6us
I20260709 13:17:10.504341 11408 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:10.506812 11445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:10.507627 11445 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 19: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:10.508175 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:10.511729 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:10.513000 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.516885 11437 raft_consensus.cc:2393] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:10.535844 11517 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
W20260709 13:17:10.518544 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 209402us
W20260709 13:17:10.527948 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 206481us
I20260709 13:17:10.539423 12033 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:10.543303 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:10.543936 11366 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
I20260709 13:17:10.545183 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:10.527465 17443 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:10.548702 17443 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.549028 17443 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:10.553712 11366 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:10.554283 17348 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:10.554983 16578 consensus_queue.cc:1048] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:10.556136 17348 raft_consensus.cc:493] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.558538 17348 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.554646 17427 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:10.562500 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:10.562903 11442 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:10.571506 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:10.573625 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:10.574440 12062 raft_consensus.cc:2393] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:10.564679 17395 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:17:10.577265 17429 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:10.577828 17429 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.581362 17427 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:10.581477 11445 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "857937fa61dc400a81345944f0a14511"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:41514
I20260709 13:17:10.582065 11445 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.582396 11445 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:10.581881 17427 raft_consensus.cc:493] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.583940 17427 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.585580 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:10.586192 11440 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 14.
I20260709 13:17:10.588021 17429 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:10.589963 17348 raft_consensus.cc:515] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.591248 12063 raft_consensus.cc:2368] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15.
I20260709 13:17:10.598730 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:10.609328 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 62244.5us
W20260709 13:17:10.610817 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 58216.1us
I20260709 13:17:10.610123 11335 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:10.612612 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 46306.6us
W20260709 13:17:10.614776 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 108681us
I20260709 13:17:10.609923 11437 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:10.619545 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.620123 11442 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:10.620306 11517 raft_consensus.cc:2393] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 in current term 14: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:10.621379 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:10.624114 11365 raft_consensus.cc:2410] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:17:10.629050 11483 leader_election.cc:400] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15.
I20260709 13:17:10.629626 11483 leader_election.cc:403] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:10.630229 16578 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.646366 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:10.647728 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:10.641763 17388 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.617805 17269 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:17:10.654444 16803 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:10.626014 17254 consensus_queue.cc:1048] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:17:10.627200 17434 consensus_queue.cc:1048] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:17:10.627451 17193 consensus_queue.cc:1048] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:17:10.697414 16578 raft_consensus.cc:2749] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15.
I20260709 13:17:10.706573 11445 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.658238 17428 consensus_queue.cc:1048] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:17:10.780081 11445 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:10.690608 17388 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:10.756182 17418 log.cc:927] Time spent T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:17:10.692847 17308 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
W20260709 13:17:10.808372 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 145216us
W20260709 13:17:10.793785 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 99415.4us
I20260709 13:17:10.810196 17448 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:10.852064 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:10.853343 11361 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:10.870033 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 233719us
I20260709 13:17:10.891487 11444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:10.892421 11444 raft_consensus.cc:3060] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:10.902243 11361 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
W20260709 13:17:10.906368 11333 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 209756us
I20260709 13:17:10.903741 11409 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:10.909857 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 126793us
I20260709 13:17:10.870815 17348 leader_election.cc:290] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:10.919143 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 156338us
I20260709 13:17:10.921208 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c753736d0d7f4578b8935b855a6b36e3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:10.911929 17278 raft_consensus.cc:2804] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:10.928139 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "857937fa61dc400a81345944f0a14511" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:10.928787 12063 raft_consensus.cc:3055] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:10.929097 12063 raft_consensus.cc:740] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:10.920161 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:10.920321 17254 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.944299 11364 raft_consensus.cc:1240] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:10.929818 12063 consensus_queue.cc:260] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.945955 12063 raft_consensus.cc:3060] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:10.950503 11512 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:10.953280 12032 leader_election.cc:304] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:10.961513 17455 log.cc:927] Time spent T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
W20260709 13:17:10.962153 17457 log.cc:927] Time spent T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:17:10.967027 17459 log.cc:927] Time spent T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:17:10.968340 17461 log.cc:927] Time spent T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:17:10.970953 17454 log.cc:927] Time spent T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:17:10.955135 17412 log.cc:927] Time spent T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:17:10.974260 17456 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:17:10.922937 17443 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:10.976437 11441 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:17:10.977341 17443 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:10.942982 17278 raft_consensus.cc:697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
W20260709 13:17:10.984114 17425 log.cc:927] Time spent T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:17:10.985652 17388 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:10.986174 17427 raft_consensus.cc:515] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.987390 12065 raft_consensus.cc:2368] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:17:10.989296 17468 raft_consensus.cc:2749] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:10.990795 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:10.991966 16811 raft_consensus.cc:2804] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:10.992249 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:10.992652 11368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:10.953341 17254 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.983079 17278 consensus_queue.cc:237] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:10.997922 11444 raft_consensus.cc:2468] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
W20260709 13:17:11.092348 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 87276.7us
I20260709 13:17:11.019986 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:11.096196 17453 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.096808 17453 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:11.097148 17427 leader_election.cc:290] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:11.098845 17453 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:11.101166 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:11.102056 11518 raft_consensus.cc:2410] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:17:11.108314 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 88410.2us
I20260709 13:17:10.995393 17254 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:10.990808 17468 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.110107 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:11.112984 11365 raft_consensus.cc:2410] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:17:11.108350 12063 raft_consensus.cc:2468] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:11.119345 11409 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:11.124354 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 172232us
I20260709 13:17:11.122392 16811 raft_consensus.cc:697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:11.168118 16811 consensus_queue.cc:237] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.164215 17465 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:17:11.210397 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:11.118667 17468 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.213639 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:11.217054 11512 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:11.221980 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:11.235262 17472 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.236016 17472 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.237155 17448 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:11.237779 17448 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:11.238315 17472 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:11.239843 16811 raft_consensus.cc:993] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: : Instructing follower 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:17:11.241247 12066 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 17.
I20260709 13:17:11.240370 16811 raft_consensus.cc:1081] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 LEADER]: Signalling peer 34a42aa6aa0a4778a532c6433d70b981 to start an election
I20260709 13:17:11.287601 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:11.283277 11512 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:11.297202 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:11.300066 12033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.287835 17478 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:17:11.308343 12032 leader_election.cc:304] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:11.296360 17484 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:11.317258 11512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
 from {username='slave'} at 127.0.0.1:48448
I20260709 13:17:11.319775 11512 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:11.320287 11512 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:11.303359 17427 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:11.322995 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:11.311724 17467 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:11.321257 16811 raft_consensus.cc:2804] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:11.319336 17484 raft_consensus.cc:493] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:11.339936 11512 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.336710 17427 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.342927 17467 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.344997 11512 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:11.345566 17427 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:17:11.345731 17467 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:11.347293 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 151890us
I20260709 13:17:11.348572 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:11.347671 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:11.357434 16811 raft_consensus.cc:697] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:11.358518 16811 consensus_queue.cc:237] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.360889 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:11.363936 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:11.367920 17468 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:11.368561 17468 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:11.343295 17484 raft_consensus.cc:3060] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:11.378300 11518 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:11.393548 11483 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: b1906ab37ba541f493936685a808c8a9
I20260709 13:17:11.398252 16874 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:11.398802 16874 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:11.399688 16874 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.410120 17484 raft_consensus.cc:515] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:11.357823 17342 raft_consensus.cc:993] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:11.412189 17342 raft_consensus.cc:1081] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:11.410323 17448 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:11.420537 17448 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:11.421257 17484 leader_election.cc:290] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:11.422022 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:11.422724 11441 raft_consensus.cc:2393] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 19: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:11.433090 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:11.433722 12065 raft_consensus.cc:3055] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:17:11.434005 12065 raft_consensus.cc:740] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:11.434784 12065 consensus_queue.cc:260] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.438472 12065 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:11.444566 11444 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 17. Current term is 18. Ops: []
I20260709 13:17:11.446400 16578 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:17:11.466285 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 237264us
I20260709 13:17:11.472555 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:11.475539 12065 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
W20260709 13:17:11.476857 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 357591us
I20260709 13:17:11.476898 11483 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:11.477873 16578 raft_consensus.cc:2804] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:11.478338 16578 raft_consensus.cc:697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 19 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:11.485700 16578 consensus_queue.cc:237] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.478614 17465 raft_consensus.cc:2749] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:11.505162 11409 leader_election.cc:400] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:17:11.505501 11409 leader_election.cc:403] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:11.507057 17213 raft_consensus.cc:3060] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:17:11.515887 12064 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:11.519361 17428 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 13:17:11.533263 17481 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.233s	user 0.001s	sys 0.001s
I20260709 13:17:11.539429 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:11.546607 11365 raft_consensus.cc:1275] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:17:11.549214 11409 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:11.556880 17479 log.cc:927] Time spent T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 13:17:11.576824 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:11.577502 11369 raft_consensus.cc:3060] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:11.625181 11365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3368fb0be64c3b92da009af48a1854" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:11.552161 17144 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:17:11.647284 11365 raft_consensus.cc:2468] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
I20260709 13:17:11.648984 17213 raft_consensus.cc:2749] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891) with higher term. Message: Invalid argument: T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for earlier term 14. Current term is 15.
I20260709 13:17:11.657989 11369 raft_consensus.cc:2468] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
I20260709 13:17:11.665978 17445 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:11.666591 17445 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.669229 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:11.670463 11436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:11.670763 11438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:41514
I20260709 13:17:11.673327 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:11.674288 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47922bb832834e72bc79e1e8159e6c1f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:11.675230 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:11.669380 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 73527.4us
I20260709 13:17:11.677641 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:11.677800 11438 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:11.678282 11438 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:11.687074 17485 log.cc:927] Time spent T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.454s	user 0.001s	sys 0.001s
I20260709 13:17:11.669240 17445 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:11.698223 17302 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:11.700664 17302 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.679479 17008 consensus_queue.cc:1059] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:17:11.708904 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:11.710489 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:11.715036 17274 consensus_queue.cc:1048] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:17:11.715751 11316 auto_rebalancer.cc:629] Failed to schedule move for tablet 593d6a54177d461680000f43998f8775: Illegal state: Replica 6e87b213a4554d94825aceec352fc094 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:11.720724 11409 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:11.735507 17465 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:11.736235 11483 leader_election.cc:304] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:11.737016 17431 raft_consensus.cc:2804] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:11.720862 16874 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:17:11.737491 17431 raft_consensus.cc:697] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:11.737862 16874 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:11.738735 16874 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:11.739002 17431 consensus_queue.cc:237] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:11.739948 16874 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:11.748718 11409 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:11.754230 11443 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:11.759855 11440 raft_consensus.cc:2468] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:11.766407 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:11.769225 17302 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:11.781271 11438 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.783275 11438 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:11.784397 12069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:11.785010 12069 raft_consensus.cc:3055] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:11.785277 12069 raft_consensus.cc:740] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:11.786006 12069 consensus_queue.cc:260] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.787037 12069 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:11.789031 12062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54376 (request call id 1687) took 1127 ms. Trace:
I20260709 13:17:11.762950 17445 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:11.798136 17445 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:11.798527 17445 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:11.812275 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:11.812891 11512 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:11.818020 12069 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:11.789433 12062 rpcz_store.cc:276] 0709 13:17:10.661621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.662012 (+   391us) service_pool.cc:224] Handling call
0709 13:17:10.662404 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:10.923169 (+260765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:10.923200 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:10.926957 (+  3757us) log.cc:844] Serialized 52 byte log entry
0709 13:17:11.217784 (+290827us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:17:11.298223 (+ 80439us) log.cc:844] Serialized 14 byte log entry
0709 13:17:11.305189 (+  6966us) log.cc:844] Serialized 14 byte log entry
0709 13:17:11.305421 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.305487 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:11.786772 (+481285us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b180025f780. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:11.788838 (+  2066us) raft_consensus.cc:1685] finished
0709 13:17:11.788859 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.788979 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52786,"prepare.queue_time_us":162392,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":102772,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":53588,"spinlock_wait_cycles":6937344,"thread_start_us":560926,"threads_started":3,"wal-append.queue_time_us":360776}
I20260709 13:17:11.819851 11409 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:11.827776 11363 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:17:11.832901 17453 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:11.881850 17453 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:11.882997 17453 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.886034 17345 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:11.886216 17445 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.886602 17345 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:11.887240 11441 raft_consensus.cc:1240] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 17. Current term is 18. Ops: []
I20260709 13:17:11.888091 11512 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:11.891870 17445 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:11.881922 17436 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:11.893445 17436 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.896941 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:11.898161 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:11.898756 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:17:11.901867 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:11.912652 11408 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:11.931339 17287 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:11.931974 17287 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:11.934649 17445 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:17:11.949602 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:11.952679 17436 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:11.993592 11361 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:12.003358 11361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:12.004935 11361 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:17:12.011170 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:12.012271 11516 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 14.
I20260709 13:17:12.027653 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:11.995802 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:12.038471 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 87219us
I20260709 13:17:12.046550 17345 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:12.051920 17287 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:11.952406 16811 raft_consensus.cc:2749] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:12.065980 17168 raft_consensus.cc:3055] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:12.066584 17168 raft_consensus.cc:740] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:12.068009 17168 consensus_queue.cc:260] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:12.106904 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 77722.3us
I20260709 13:17:12.109987 11333 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
W20260709 13:17:12.110913 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 104403us
I20260709 13:17:12.125399 17222 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:12.126250 17222 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:12.126791 17222 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:12.063294 17288 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:17:12.118841 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:12.130934 11441 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:17:12.138801 11519 raft_consensus.cc:1275] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:17:12.140450 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:12.119927 17482 consensus_queue.cc:1059] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:12.142874 17482 raft_consensus.cc:3055] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:17:12.143265 17482 raft_consensus.cc:740] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:12.144600 17482 consensus_queue.cc:260] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:12.146395 17482 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:17:12.151994 17148 consensus_queue.cc:1175] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b1906ab37ba541f493936685a808c8a9" responder_term: 17 status { last_received { term: 17 index: 14 } last_committed_idx: 13 last_received_current_leader { term: 17 index: 14 } }
I20260709 13:17:12.154592 17501 raft_consensus.cc:493] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:12.160633 17501 raft_consensus.cc:515] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:12.167307 16954 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:12.168533 16954 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:12.171592 16954 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:12.190831 12062 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:12.192471 12063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:12.205066 17501 leader_election.cc:290] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:12.212008 11484 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:12.218906 17431 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:17:12.219336 17431 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b1906ab37ba541f493936685a808c8a9)
I20260709 13:17:12.219720 17431 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:12.234338 11511 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:17:12.236406 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:12.237811 11267 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 16 to 17, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 17 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.247730 17008 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:12.251745 12066 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:17:12.264660 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:12.270036 17168 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:12.277566 11512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:12.279969 11440 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:12.286520 17039 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:12.288386 11408 leader_election.cc:304] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:12.289642 17274 raft_consensus.cc:2749] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:12.291252 17039 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:12.291949 17039 consensus_queue.cc:1243] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer b1906ab37ba541f493936685a808c8a9 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:17:12.256500 17431 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:12.299880 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 235313us
I20260709 13:17:12.316825 11511 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:12.325611 17471 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:12.317854 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d370e878a484e1aaaa0df1466da6f24" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:12.314750 17148 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:17:12.323664 17222 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:12.351150 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 310597us
I20260709 13:17:12.356192 11441 raft_consensus.cc:1240] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:17:12.356813 11440 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:17:12.366586 12034 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.367759 11363 raft_consensus.cc:1275] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:12.373764 17431 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:12.374636 11262 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39444 (request call id 355) took 6058 ms. Trace:
I20260709 13:17:12.377704 11511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:12.378580 11511 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:12.375048 11262 rpcz_store.cc:276] 0709 13:17:06.316433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.319567 (+  3134us) service_pool.cc:224] Handling call
0709 13:17:12.374394 (+6054827us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:10.120497 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:10.121068 (+   571us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:10.121104 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:17:10.121124 (+    20us) tablet.cc:662] Decoding operations
0709 13:17:10.122934 (+  1810us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:10.123000 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:10.123031 (+    31us) tablet.cc:685] Acquiring locks for 28 operations
0709 13:17:10.148449 (+ 25418us) tablet.cc:701] Row locks acquired
0709 13:17:10.148464 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:17:10.148612 (+   148us) write_op.cc:270] Start()
0709 13:17:10.148735 (+   123us) write_op.cc:276] Timestamp: P: 1783603030148563 usec, L: 0
0709 13:17:10.148751 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:17:10.203701 (+ 54950us) log.cc:844] Serialized 10826 byte log entry
0709 13:17:10.577021 (+373320us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b18001dfe60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911522 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:10.580092 (+  3071us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b18001dfe60. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b98f3e07 00007f35c274eab9 00007f35b991069b 00007f35b990e93f 00007f35b990e455 00007f35bc97b190 00007f35bc974ae7 00007f35bc973dd9 00007f35bc97e930
0709 13:17:10.862312 (+282220us) op_driver.cc:464] REPLICATION: finished
0709 13:17:10.961556 (+ 99244us) write_op.cc:301] APPLY: starting
0709 13:17:10.961646 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:11.367096 (+405450us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:11.367123 (+    27us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:11.537479 (+170356us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:11.537837 (+   358us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0709 13:17:11.537891 (+    54us) write_op.cc:312] APPLY: finished
0709 13:17:11.538380 (+   489us) log.cc:844] Serialized 244 byte log entry
0709 13:17:11.538550 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:11.538771 (+   221us) write_op.cc:454] Released schema lock
0709 13:17:11.538830 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:11.550905 (+ 12075us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:11.553906 (+  3001us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7302,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":94743,"apply.run_cpu_time_us":35341,"apply.run_wall_time_us":579113,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121055,"num_ops":28,"prepare.queue_time_us":514883,"prepare.run_cpu_time_us":13611,"prepare.run_wall_time_us":358910,"raft.queue_time_us":302151,"raft.run_cpu_time_us":21246,"raft.run_wall_time_us":88398,"replication_time_us":704350,"spinlock_wait_cycles":37422592,"thread_start_us":803189,"threads_started":4,"wal-append.queue_time_us":275498}]]}
I20260709 13:17:12.375577 17222 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:12.396782 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:12.400276 11441 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:12.361476 11267 catalog_manager.cc:5697] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 13 to 15, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 15 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.404206 11267 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 13 to 16, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 16 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.395182 17302 consensus_queue.cc:1059] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:17:12.408046 11267 catalog_manager.cc:5697] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 13 to 17, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 17 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.412261 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:12.412904 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:12.413816 12062 raft_consensus.cc:1217] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 10.8->[11.9-15.11]   Dedup: 11.9->[14.10-15.11]
I20260709 13:17:12.423915 17324 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:12.419212 12062 pending_rounds.cc:85] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Aborting all ops after (but not including) 9
I20260709 13:17:12.424868 12062 pending_rounds.cc:107] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:17:12.425189 12062 raft_consensus.cc:2889] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.427367 17342 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:12.427786 17342 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
W20260709 13:17:12.424062 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 57689.4us
I20260709 13:17:12.438055 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.428836 17342 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:12.439116 17189 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:12.439880 12034 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.440788 17342 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:12.440883 17521 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:12.523856 12065 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:17:12.526284 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 45329.6us
W20260709 13:17:12.540532 11336 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.541221 16954 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Committing config change with OpId 17.10: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:12.526603 17274 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:17:12.549569 11519 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Committing config change with OpId 17.10: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:12.558585 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:12.559533 11366 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 15: Already voted for candidate 01ae50c877d04ba687e4e06259ba5581 in this term.
I20260709 13:17:12.561699 11366 raft_consensus.cc:1275] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:17:12.563148 11267 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 11 to 18, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 18 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.566442 11267 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 13 to 15, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 15 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.569860 17519 consensus_queue.cc:1048] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:12.572712 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.583240 11516 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:17:12.593254 11335 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.613955 11441 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:12.633407 17168 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:12.634682 17519 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:12.656253 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 83649.4us
I20260709 13:17:12.637583 17501 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:17:12.657222 11267 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 14 to 16, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 16 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.669477 11438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:12.671129 11438 raft_consensus.cc:2393] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 15: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
I20260709 13:17:12.675942 11335 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.655201 17222 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:17:12.673108 11267 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.666321 17039 consensus_queue.cc:1048] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:17:12.691906 11440 raft_consensus.cc:1240] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:12.697970 17518 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:12.684725 11267 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 12 to 13, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:12.718223 11438 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:12.717093 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:12.740855 11267 catalog_manager.cc:5697] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 13 to 14, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 14 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.735090 17431 raft_consensus.cc:2749] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:17:12.752727 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:12.753427 12066 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
I20260709 13:17:12.751135 11267 catalog_manager.cc:5697] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.773608 11267 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 12 to 15, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 15 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.786661 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6002d9cfb1c74b70b8ec301e31e4bbe9" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:12.769140 17484 consensus_queue.cc:1048] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:17:12.788378 12062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
W20260709 13:17:12.853060 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 112590us
I20260709 13:17:12.856943 11267 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 8 to 12, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 12 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
W20260709 13:17:12.869884 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 181195us
I20260709 13:17:12.821619 17519 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:17:12.893005 12069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ea18eac305644ed8d44618c2c0a8d0d"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:12.893952 12069 raft_consensus.cc:606] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.894300 12069 raft_consensus.cc:614] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:12.896173 17530 log.cc:927] Time spent T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:17:12.900214 12061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81760accce3a440a82323bd3ff53c551"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:12.900887 12061 raft_consensus.cc:606] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.918535 12061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8ec7d9e61ab4ff5bd59a37021d68dba"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:12.922855 12061 raft_consensus.cc:606] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:12.923205 12061 raft_consensus.cc:614] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:12.926383 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 131442us
W20260709 13:17:12.926680 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 61737.5us
I20260709 13:17:12.930516 11267 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:12.934722 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 73850.1us
I20260709 13:17:12.938201 11442 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Committing config change with OpId 17.10: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:12.949365 12061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcf62f83a1754dcc80291daa521a9c06"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:12.949957 12061 raft_consensus.cc:606] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:12.950263 12061 raft_consensus.cc:614] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:12.948290 11334 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1073b5e444bf4b34ad3dca3ea96efc76. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:12.962563 11267 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 11 to 13, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 13 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.987377 12066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4aa2e377a0f45378e0df4a788c07ddf"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:48186
I20260709 13:17:12.988014 12066 raft_consensus.cc:606] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:12.988317 12066 raft_consensus.cc:614] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:12.972755 11267 catalog_manager.cc:5697] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 9 to 13, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 13 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:12.996101 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:13.011101 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:17:13.022181 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 47497.8us
W20260709 13:17:13.024224 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 61503.2us
I20260709 13:17:13.045210 11334 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581; no voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:13.054420 17554 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:17:13.066782 17510 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 13:17:13.071805 17529 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:17:13.073035 17144 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:13.074415 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 106102us
I20260709 13:17:13.075804 17144 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.082398 17144 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:13.087178 17547 log.cc:927] Time spent T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: Append to log took a long time: real 0.097s	user 0.003s	sys 0.000s
I20260709 13:17:13.095456 12062 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 15.
W20260709 13:17:13.098347 17534 log.cc:927] Time spent T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:13.157922 17539 raft_consensus.cc:993] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:13.181221 17539 raft_consensus.cc:1081] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:13.207204 11512 raft_consensus.cc:1275] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:17:13.218963 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 53543.3us
I20260709 13:17:13.231958 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:13.263559 17548 consensus_queue.cc:1048] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:17:13.240713 17554 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:13.267715 17554 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.269814 17554 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:13.271538 17559 log.cc:927] Time spent T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:17:13.278163 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:13.278985 11516 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 18.
I20260709 13:17:13.286665 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 
I20260709 13:17:13.289546 17222 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:13.290001 17222 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:13.290373 17222 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:17:13.300107 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 153684us
W20260709 13:17:13.304560 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 98607us
W20260709 13:17:13.342298 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 107393us
I20260709 13:17:13.382174 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:13.375430 11440 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81760accce3a440a82323bd3ff53c551"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:41514
I20260709 13:17:13.389734 11440 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.390107 11440 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:13.393440 11363 raft_consensus.cc:2410] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e87b213a4554d94825aceec352fc094 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:17:13.438254 11409 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 46253.1us
I20260709 13:17:13.440448 17473 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:13.442811 17473 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.445062 17473 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:13.454028 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:13.454866 11437 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 18: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:13.463604 17222 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.471851 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:13.472570 11437 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
W20260709 13:17:13.472790 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 111610us
I20260709 13:17:13.476173 17563 ts_tablet_manager.cc:933] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
I20260709 13:17:13.476945 17302 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:13.479367 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:13.481678 11440 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:13.484586 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 46934.7us
I20260709 13:17:13.489768 11443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:13.490484 11443 raft_consensus.cc:2393] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01ae50c877d04ba687e4e06259ba5581 in current term 18: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:13.491214 17302 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:13.493644 17302 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:13.497200 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:13.508037 11363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:13.508939 11440 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:13.502835 11366 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:17:13.517411 11517 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 18.
I20260709 13:17:13.514616 11409 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:13.484988 17222 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:13.522861 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 58020.2us
I20260709 13:17:13.537890 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:13.538631 11366 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:13.554553 11366 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:13.554894 11440 raft_consensus.cc:1240] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Rejecting Update request from peer b1906ab37ba541f493936685a808c8a9 for earlier term 13. Current term is 14. Ops: []
I20260709 13:17:13.557318 17302 consensus_queue.cc:1059] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:13.558369 17302 raft_consensus.cc:3055] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:17:13.558863 17302 raft_consensus.cc:740] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:13.559844 17302 consensus_queue.cc:260] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:13.561251 17302 raft_consensus.cc:3060] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:13.565502 11409 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:13.566323 17274 raft_consensus.cc:2804] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:13.569504 17274 raft_consensus.cc:697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:13.570569 11333 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:13.570881 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:13.572680 17274 consensus_queue.cc:237] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:13.579577 17473 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:13.581575 17473 raft_consensus.cc:493] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:13.581983 17473 raft_consensus.cc:3060] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:13.582625 17395 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:17:13.583168 17395 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:13.584015 17395 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.602583 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81760accce3a440a82323bd3ff53c551" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:13.609723 17571 raft_consensus.cc:2749] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:13.606215 17563 tablet_copy_client.cc:323] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.65:42857
I20260709 13:17:13.615840 11380 tablet_copy_service.cc:140] P 01ae50c877d04ba687e4e06259ba5581: Received BeginTabletCopySession request for tablet 1073b5e444bf4b34ad3dca3ea96efc76 from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:47990)
I20260709 13:17:13.616394 11380 tablet_copy_service.cc:161] P 01ae50c877d04ba687e4e06259ba5581: Beginning new tablet copy session on tablet 1073b5e444bf4b34ad3dca3ea96efc76 from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:47990: session id = b1906ab37ba541f493936685a808c8a9-1073b5e444bf4b34ad3dca3ea96efc76
I20260709 13:17:13.662484 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:13.689901 17473 raft_consensus.cc:515] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.704104 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:13.704941 11437 raft_consensus.cc:2393] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 14: Already voted for candidate 6e87b213a4554d94825aceec352fc094 in this term.
I20260709 13:17:13.705040 11380 tablet_copy_source_session.cc:215] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:13.708544 17563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1073b5e444bf4b34ad3dca3ea96efc76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:13.709024 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd69c97ea4640a0a71117011ab4e5d4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:13.699262 17473 leader_election.cc:290] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:13.717620 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:13.719630 17552 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:13.725108 12066 raft_consensus.cc:2468] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:13.729511 11370 raft_consensus.cc:2468] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:17:13.765703 11483 leader_election.cc:304] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:13.769214 17568 raft_consensus.cc:2804] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:13.769836 17568 raft_consensus.cc:697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:13.770735 17568 consensus_queue.cc:237] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.792629 17473 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.793141 17473 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:13.795292 17473 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:13.798202 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:13.798849 11437 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 18.
I20260709 13:17:13.799963 11484 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:13.797467 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:13.801138 12066 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:13.804908 17482 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:13.805317 17482 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.805642 17482 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:13.792093 17563 tablet_copy_client.cc:806] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:13.807859 17563 tablet_copy_client.cc:670] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:13.812587 17482 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:13.819283 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:13.819953 12065 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:17:13.821352 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf199b70fbb94d728a09194d18ca3b1e" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:13.821931 11437 raft_consensus.cc:3060] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:13.819497 17563 tablet_copy_client.cc:538] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:13.832621 12065 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
I20260709 13:17:13.833173 17482 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:13.841827 11483 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:13.866361 17563 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:17:13.867139 17560 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.871939 11437 raft_consensus.cc:2468] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 19.
I20260709 13:17:13.877046 17482 raft_consensus.cc:2804] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:13.888145 17482 raft_consensus.cc:697] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 19 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:13.884101 17560 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:13.937362 17560 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:13.937983 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:13.885141 11268 catalog_manager.cc:5697] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 17 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:13.939620 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:13.940310 11519 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 18.
I20260709 13:17:13.942090 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:13.926244 17482 consensus_queue.cc:237] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:13.943430 17560 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:13.947353 17560 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.947803 17560 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:13.941174 11268 catalog_manager.cc:5697] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 17 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:13.953238 11268 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 18 to 19, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 19 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.956210 11268 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 8 to 11, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 11 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:13.957551 11268 catalog_manager.cc:5697] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 12, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.959595 11268 catalog_manager.cc:5697] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 15, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 15 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:13.961112 11268 catalog_manager.cc:5697] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 9 to 10, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 10 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:13.962458 11268 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 15 to 16, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 16 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.965766 11267 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56588 (request call id 240) took 4877 ms. Trace:
I20260709 13:17:13.965327 11268 catalog_manager.cc:5697] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.968209 11268 catalog_manager.cc:5697] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 13, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 13 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.972201 11268 catalog_manager.cc:5697] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 12, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.974772 12066 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:17:13.974150 11268 catalog_manager.cc:5697] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 7 to 12, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.976163 17144 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:13.975904 11268 catalog_manager.cc:5697] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 19, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 19 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:13.966229 11267 rpcz_store.cc:276] 0709 13:17:09.088500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.088883 (+   383us) service_pool.cc:224] Handling call
0709 13:17:13.965701 (+4876818us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:13.383481 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:13.384152 (+   671us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:13.384214 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:17:13.384234 (+    20us) tablet.cc:662] Decoding operations
0709 13:17:13.447261 (+ 63027us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:13.447370 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:13.447413 (+    43us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:17:13.448619 (+  1206us) tablet.cc:701] Row locks acquired
0709 13:17:13.448665 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:17:13.448832 (+   167us) write_op.cc:270] Start()
0709 13:17:13.448976 (+   144us) write_op.cc:276] Timestamp: P: 1783603033448768 usec, L: 0
0709 13:17:13.449010 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:17:13.449736 (+   726us) log.cc:844] Serialized 6182 byte log entry
0709 13:17:13.557536 (+107800us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b1800042c40. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b99113f3 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:13.567549 (+ 10013us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b1800042c40. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c27d9de1 00007f35c27e65b1 00007f35b9911557 00007f35b99137c1 00007f35b9913779 00007f35b9913709 00007f35b99136d1 00007f35b99129cd 000000000038f354 000000000038f199 00007f35b98f483a
0709 13:17:13.703182 (+135633us) op_driver.cc:464] REPLICATION: finished
0709 13:17:13.750729 (+ 47547us) write_op.cc:301] APPLY: starting
0709 13:17:13.776870 (+ 26141us) spinlock_profiling.cc:243] Waited 7.65 ms on lock 0x7b5400189628. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35c2dba9e1 00007f35c2db964c 00007f35c0449e10 00007f35c043597c 00007f35c0439d61 00007f35c0439d19 00007f35c0439ca9 00007f35c0439c71 00007f35c0438e9d 000000000038f354 000000000038f199
0709 13:17:13.777009 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:13.792088 (+ 15079us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:13.792124 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:13.856809 (+ 64685us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:13.857150 (+   341us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:17:13.857192 (+    42us) write_op.cc:312] APPLY: finished
0709 13:17:13.857579 (+   387us) log.cc:844] Serialized 148 byte log entry
0709 13:17:13.857769 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:13.857914 (+   145us) write_op.cc:454] Released schema lock
0709 13:17:13.857986 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:13.863288 (+  5302us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:13.863318 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5403,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":47286,"apply.run_cpu_time_us":22839,"apply.run_wall_time_us":112517,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70943,"num_ops":16,"prepare.queue_time_us":94674,"prepare.run_cpu_time_us":28642,"prepare.run_wall_time_us":159905,"raft.queue_time_us":108250,"raft.run_cpu_time_us":4037,"raft.run_wall_time_us":16367,"replication_time_us":253966,"spinlock_wait_cycles":67289984,"thread_start_us":248065,"threads_started":4,"wal-append.queue_time_us":93469}]]}
I20260709 13:17:14.043933 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:14.030843 17560 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:14.108745 11517 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:17:14.112396 11370 raft_consensus.cc:1275] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:17:14.112668 17582 log.cc:927] Time spent T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:17:14.113741 17222 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:14.119633 17571 consensus_queue.cc:1048] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:14.125437 17560 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:14.130592 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:14.135097 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:14.199832 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:14.202148 17571 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:17:14.220713 11440 raft_consensus.cc:1240] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Rejecting Update request from peer 01ae50c877d04ba687e4e06259ba5581 for earlier term 18. Current term is 19. Ops: []
I20260709 13:17:14.226203 17554 consensus_queue.cc:1059] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:14.226927 17222 raft_consensus.cc:3055] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:17:14.227272 17222 raft_consensus.cc:740] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:14.228148 17222 consensus_queue.cc:260] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:14.241065 17222 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:14.318684 17473 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.319264 17473 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:14.322427 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:14.323014 11366 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:14.325191 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:14.325769 12067 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:14.331127 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:14.332326 17550 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:14.332757 17550 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.333106 17550 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.334018 17563 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:14.337971 11267 catalog_manager.cc:5697] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 14, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 14 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:14.322887 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:14.343176 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:14.343529 17563 tablet_bootstrap.cc:492] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:17:14.344432 17563 ts_tablet_manager.cc:1403] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.478s	user 0.047s	sys 0.025s
I20260709 13:17:14.343811 11267 catalog_manager.cc:5697] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:14.345989 17473 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:14.352195 11267 catalog_manager.cc:5697] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:14.353557 11267 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 17 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:14.358908 17550 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:14.360719 17563 raft_consensus.cc:359] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:17:14.368512 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:14.369129 12067 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.370277 17550 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:14.362406 11370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:14.366657 11440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:14.375751 11370 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.375821 11440 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:14.380875 12067 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:14.369055 17563 raft_consensus.cc:740] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:17:14.382362 11370 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:14.381995 17563 consensus_queue.cc:260] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:17:14.386922 11483 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:14.387753 17473 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:14.388193 12067 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:17:14.388242 17473 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:14.396003 17473 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:14.408142 11266 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39438 (request call id 362) took 17997 ms (client timeout 15000 ms). Trace:
W20260709 13:17:14.408581 11266 rpcz_store.cc:269] 0709 13:16:56.410570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.416580 (+  6010us) service_pool.cc:224] Handling call
0709 13:17:14.407944 (+17991364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33,"reported_tablets":46,"spinlock_wait_cycles":12544}
I20260709 13:17:14.418123 11437 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:14.420809 17568 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:14.424630 17550 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:14.430841 11440 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:14.436180 11370 raft_consensus.cc:1275] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:17:14.437680 17484 consensus_queue.cc:1048] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:14.389240 17563 ts_tablet_manager.cc:1434] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:17:14.441951 11380 tablet_copy_service.cc:342] P 01ae50c877d04ba687e4e06259ba5581: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-1073b5e444bf4b34ad3dca3ea96efc76 received from {username='slave'} at 127.0.0.1:47990
I20260709 13:17:14.442411 11380 tablet_copy_service.cc:434] P 01ae50c877d04ba687e4e06259ba5581: ending tablet copy session b1906ab37ba541f493936685a808c8a9-1073b5e444bf4b34ad3dca3ea96efc76 on tablet 1073b5e444bf4b34ad3dca3ea96efc76 with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:17:14.460685 12066 raft_consensus.cc:1217] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.9->[17.10-17.10]   Dedup: 17.10->[]
I20260709 13:17:14.485766 11266 catalog_manager.cc:5697] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:14.520255 11440 raft_consensus.cc:1275] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:17:14.523525 17568 consensus_queue.cc:1048] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:14.574877 11262 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:14.578166 11262 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 13 to 16, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 16 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:14.584556 11262 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 15 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 17 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:14.585975 11262 catalog_manager.cc:5697] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 14 to 18, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 18 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:14.587801 11262 catalog_manager.cc:5697] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:14.627653 11505 tablet_service.cc:1558] Processing DeleteTablet for tablet 857937fa61dc400a81345944f0a14511 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 8)) from {username='slave'} at 127.0.0.1:48450
I20260709 13:17:14.637497 11252 catalog_manager.cc:5028] TS 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): tablet 857937fa61dc400a81345944f0a14511 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
W20260709 13:17:14.677197 11263 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39460 (request call id 367) took 16128 ms (client timeout 15000 ms). Trace:
W20260709 13:17:14.677611 11263 rpcz_store.cc:269] 0709 13:16:58.548650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.548872 (+   222us) service_pool.cc:224] Handling call
0709 13:17:14.676999 (+16128127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10990,"reported_tablets":46}
I20260709 13:17:14.681622 11266 catalog_manager.cc:5697] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 16 to 19, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 19 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:14.704094 11264 catalog_manager.cc:5697] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 16 to 19, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 19 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:14.717080 11262 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39438 (request call id 363) took 1735 ms. Trace:
I20260709 13:17:14.730890 11262 rpcz_store.cc:276] 0709 13:17:12.981342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.981560 (+   218us) service_pool.cc:224] Handling call
0709 13:17:14.716996 (+1735436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:14.613888 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:14.614381 (+   493us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:14.614415 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:17:14.614428 (+    13us) tablet.cc:662] Decoding operations
0709 13:17:14.615027 (+   599us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:14.615077 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:14.615105 (+    28us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:17:14.615622 (+   517us) tablet.cc:701] Row locks acquired
0709 13:17:14.615640 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:17:14.615772 (+   132us) write_op.cc:270] Start()
0709 13:17:14.615873 (+   101us) write_op.cc:276] Timestamp: P: 1783603034615732 usec, L: 0
0709 13:17:14.615894 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:17:14.616454 (+   560us) log.cc:844] Serialized 2044 byte log entry
0709 13:17:14.618375 (+  1921us) op_driver.cc:464] REPLICATION: finished
0709 13:17:14.618776 (+   401us) write_op.cc:301] APPLY: starting
0709 13:17:14.618844 (+    68us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:14.622507 (+  3663us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:14.622604 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:14.625514 (+  2910us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:14.625686 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:17:14.625744 (+    58us) write_op.cc:312] APPLY: finished
0709 13:17:14.625979 (+   235us) log.cc:844] Serialized 57 byte log entry
0709 13:17:14.626725 (+   746us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:14.626826 (+   101us) write_op.cc:454] Released schema lock
0709 13:17:14.626865 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:14.627209 (+   344us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:14.627250 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46708,"reported_tablets":46,"spinlock_wait_cycles":286592,"child_traces":[["op",{"apply.queue_time_us":218,"apply.run_cpu_time_us":6843,"apply.run_wall_time_us":8378,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20880,"num_ops":5,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":2821,"prepare.run_wall_time_us":2820,"replication_time_us":2382}]]}
I20260709 13:17:14.858608 11264 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39460 (request call id 368) took 1018 ms. Trace:
I20260709 13:17:14.867353 11437 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:14.869465 17568 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:17:14.859032 11264 rpcz_store.cc:276] 0709 13:17:13.839722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.839953 (+   231us) service_pool.cc:224] Handling call
0709 13:17:14.858526 (+1018573us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:14.745968 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:14.746443 (+   475us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:14.746476 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:17:14.746489 (+    13us) tablet.cc:662] Decoding operations
0709 13:17:14.746804 (+   315us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:14.746857 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:14.746882 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:14.747045 (+   163us) tablet.cc:701] Row locks acquired
0709 13:17:14.747057 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:17:14.747184 (+   127us) write_op.cc:270] Start()
0709 13:17:14.747290 (+   106us) write_op.cc:276] Timestamp: P: 1783603034747144 usec, L: 0
0709 13:17:14.747311 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:17:14.751034 (+  3723us) log.cc:844] Serialized 530 byte log entry
0709 13:17:14.752809 (+  1775us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b54001897d4. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35bc967571 00007f35bc9f9e28 00007f35c0434581 00007f35c0433d58 00007f35c04389f1 00007f35c04389a9 00007f35c0438939 00007f35c0438901 00007f35c0437b2d 000000000038f354 000000000038f199
0709 13:17:14.754424 (+  1615us) op_driver.cc:464] REPLICATION: finished
0709 13:17:14.754831 (+   407us) write_op.cc:301] APPLY: starting
0709 13:17:14.754897 (+    66us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:14.755665 (+   768us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:14.755685 (+    20us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:14.756322 (+   637us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:14.756456 (+   134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:17:14.756497 (+    41us) write_op.cc:312] APPLY: finished
0709 13:17:14.756659 (+   162us) log.cc:844] Serialized 25 byte log entry
0709 13:17:14.756816 (+   157us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:14.756891 (+    75us) write_op.cc:454] Released schema lock
0709 13:17:14.756926 (+    35us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:14.757200 (+   274us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:14.757227 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35642,"reported_tablets":46,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":230,"apply.run_cpu_time_us":2353,"apply.run_wall_time_us":2353,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4176,"num_ops":1,"prepare.queue_time_us":123,"prepare.run_cpu_time_us":3896,"prepare.run_wall_time_us":6984,"replication_time_us":6857,"spinlock_wait_cycles":5230336}]]}
I20260709 13:17:14.886353 11264 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 14 to 16, leader changed from b1906ab37ba541f493936685a808c8a9 (127.10.4.68) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 16 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.888746 11264 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 17 to 18, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 18 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } }
I20260709 13:17:14.903486 11370 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:14.933935 17484 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:14.936946 12066 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:17:15.012748 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 284201us
W20260709 13:17:15.013119 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 55200.3us
W20260709 13:17:15.013681 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49301.6us
I20260709 13:17:15.021054 17473 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:15.251379 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:15.336637 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 61235.5us
W20260709 13:17:15.340996 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 64132.2us
I20260709 13:17:15.345796 17602 raft_consensus.cc:1064] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:17:15.351822 17602 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 8, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.382236 12066 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 LEARNER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:15.384286 17611 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:15.395955 11517 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:15.398659 11437 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:15.401674 17554 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:15.404057 17554 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:15.436343 17609 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.438195 11437 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.444144 11516 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.479390 12066 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.490869 11262 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.510802 17605 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.511456 17605 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:15.514174 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:15.514767 11437 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 19.
I20260709 13:17:15.516137 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:15.516892 17605 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:15.517433 17555 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:17:15.518507 17555 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.518815 17555 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:15.517072 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:15.528580 17555 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:15.538733 17555 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 20 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:15.538920 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:15.545713 11441 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:15.539758 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:15.553668 11441 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 20.
I20260709 13:17:15.554849 11335 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:15.555750 17605 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:17:15.557695 17605 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 20 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:15.558667 17605 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:15.610370 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:38026
I20260709 13:17:15.610967 11516 raft_consensus.cc:606] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:17:15.613857 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:38026
I20260709 13:17:15.614382 11516 raft_consensus.cc:606] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:17:15.618270 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:38026
I20260709 13:17:15.618835 11516 raft_consensus.cc:606] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:17:15.622560 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a38969de5c824718aa81b9622624bc51"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:38026
I20260709 13:17:15.623013 11516 raft_consensus.cc:606] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:15.624150 17568 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:15.624676 17568 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:15.626746 17568 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:15.627761 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:15.628456 11437 raft_consensus.cc:2368] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 19. Current term is 20.
I20260709 13:17:15.630344 11484 leader_election.cc:400] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 19. Current term is 20.
I20260709 13:17:15.630646 11484 leader_election.cc:403] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:15.631227 17568 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:17:15.631801 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01ae50c877d04ba687e4e06259ba5581" is_pre_election: true
I20260709 13:17:15.641893 17568 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257) with higher term. Message: Invalid argument: T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 for earlier term 19. Current term is 20.
I20260709 13:17:15.642316 11516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:38026
I20260709 13:17:15.642894 11516 raft_consensus.cc:606] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Received request to transfer leadership to TS b1906ab37ba541f493936685a808c8a9
I20260709 13:17:15.644066 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:15.645709 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:15.645959 11369 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.652750 17484 raft_consensus.cc:993] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:15.653168 17550 raft_consensus.cc:993] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:15.653643 17550 raft_consensus.cc:1081] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:15.657531 11516 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:15.657845 12066 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:17:15.660131 17555 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:15.661039 17609 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:15.667466 12066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:17:15.667968 12066 raft_consensus.cc:493] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:15.668283 12066 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.674988 11519 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.679343 12065 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.673375 17555 raft_consensus.cc:2955] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.676015 11264 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.686391 12066 raft_consensus.cc:515] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.689056 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:15.689530 11369 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.696468 11369 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:15.706471 12033 leader_election.cc:304] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:15.707288 17612 raft_consensus.cc:2804] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:15.707979 17612 raft_consensus.cc:697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:15.708775 17612 consensus_queue.cc:237] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.715373 17568 raft_consensus.cc:1081] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:15.717383 11437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a38969de5c824718aa81b9622624bc51"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51474
I20260709 13:17:15.717885 11437 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:15.718240 11437 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.723734 11369 raft_consensus.cc:1240] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 15. Current term is 16. Ops: []
I20260709 13:17:15.725052 17568 consensus_queue.cc:1059] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:15.726444 11437 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:15.728048 17484 raft_consensus.cc:3055] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:15.728432 17484 raft_consensus.cc:740] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:15.729893 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:15.730468 11369 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.730067 17484 consensus_queue.cc:260] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.731115 17484 raft_consensus.cc:3060] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.730628 11264 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: config changed from index 11 to 12, VOTER 6e87b213a4554d94825aceec352fc094 (127.10.4.66) evicted. New cstate: current_term: 17 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:15.743868 11253 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1073b5e444bf4b34ad3dca3ea96efc76 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:15.744879 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38969de5c824718aa81b9622624bc51" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:15.745435 11519 raft_consensus.cc:3055] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:15.745712 11519 raft_consensus.cc:740] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:15.746335 11519 consensus_queue.cc:260] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:15.747359 11519 raft_consensus.cc:3060] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.753464 11369 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:17:15.742177 11437 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:15.755918 11409 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:15.757184 17144 raft_consensus.cc:2804] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:15.757678 17144 raft_consensus.cc:697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 15 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:15.758474 17144 consensus_queue.cc:237] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:15.774709 11268 catalog_manager.cc:5697] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 15 to 16, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 16 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.776430 11519 raft_consensus.cc:2468] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 15.
I20260709 13:17:15.796871 12066 leader_election.cc:290] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:15.798163 11433 tablet_service.cc:1558] Processing DeleteTablet for tablet 1073b5e444bf4b34ad3dca3ea96efc76 with delete_type TABLET_DATA_TOMBSTONED (TS 6e87b213a4554d94825aceec352fc094 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:55822
I20260709 13:17:15.802076 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbbdba17cce24fe7923cf755be2d3e0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:15.809928 11519 raft_consensus.cc:2468] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:15.817626 17610 raft_consensus.cc:993] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:15.828359 17610 raft_consensus.cc:1081] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:15.837932 12066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:17:15.838632 12066 raft_consensus.cc:493] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:15.838961 12066 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.863687 11266 catalog_manager.cc:5697] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 14 to 15, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 15 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.865048 12066 raft_consensus.cc:515] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.866931 12066 leader_election.cc:290] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:15.867774 11369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:15.868314 11369 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.863597 17625 tablet_replica.cc:333] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:15.873566 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988262f8a4a846be8b350d9d99c4a730" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:15.878067 11519 raft_consensus.cc:3055] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:15.878410 11519 raft_consensus.cc:740] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:15.879092 11519 consensus_queue.cc:260] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.880371 11519 raft_consensus.cc:3060] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:15.884828 17625 raft_consensus.cc:2243] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:15.899362 17625 raft_consensus.cc:2272] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:15.901474 11519 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:15.895077 11369 raft_consensus.cc:2468] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:15.906044 12033 leader_election.cc:304] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:15.906822 17612 raft_consensus.cc:2804] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:15.907266 17612 raft_consensus.cc:697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:15.908098 17612 consensus_queue.cc:237] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.904421 17625 ts_tablet_manager.cc:1916] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:15.942521 17568 raft_consensus.cc:993] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:15.943012 17568 raft_consensus.cc:1081] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:15.943976 11264 catalog_manager.cc:5697] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 15 to 16, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 16 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.948468 12066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:17:15.949048 12066 raft_consensus.cc:493] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:15.949365 12066 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.961169 17625 ts_tablet_manager.cc:1929] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.11
I20260709 13:17:15.963685 12066 raft_consensus.cc:515] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.966737 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:15.967355 11441 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.976740 11441 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:15.984539 12066 leader_election.cc:290] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:15.984905 11437 raft_consensus.cc:1240] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Rejecting Update request from peer 34a42aa6aa0a4778a532c6433d70b981 for earlier term 14. Current term is 15. Ops: []
I20260709 13:17:15.986380 12034 leader_election.cc:304] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:15.991197 17612 raft_consensus.cc:2804] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:17:15.991780 17612 raft_consensus.cc:697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:15.992173 17610 consensus_queue.cc:1059] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:15.993237 17610 raft_consensus.cc:3055] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:17:15.993573 17610 raft_consensus.cc:740] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:15.994351 17610 consensus_queue.cc:260] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.992614 17612 consensus_queue.cc:237] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:15.996451 17610 raft_consensus.cc:3060] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:17:15.986444 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64ec4a2bcc7f4fe1851f6d77e7515fed" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:15.963992 17625 log.cc:1199] T 1073b5e444bf4b34ad3dca3ea96efc76 P 6e87b213a4554d94825aceec352fc094: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/wals/1073b5e444bf4b34ad3dca3ea96efc76
I20260709 13:17:16.006953 11254 catalog_manager.cc:5028] TS 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): tablet 1073b5e444bf4b34ad3dca3ea96efc76 (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:17:16.021056 11519 raft_consensus.cc:2468] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:16.027456 11264 catalog_manager.cc:5697] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 14 to 15, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 15 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:16.079818 11519 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:17:16.081256 17222 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:16.124827 11437 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:17:16.126796 17602 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:16.188920 17568 raft_consensus.cc:993] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:16.189428 17610 raft_consensus.cc:1081] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Signalling peer b1906ab37ba541f493936685a808c8a9 to start an election
I20260709 13:17:16.191004 12066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3"
dest_uuid: "b1906ab37ba541f493936685a808c8a9"
 from {username='slave'} at 127.0.0.1:54376
I20260709 13:17:16.191550 12066 raft_consensus.cc:493] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.191879 12066 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.198580 12066 raft_consensus.cc:515] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:16.206194 11366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
I20260709 13:17:16.208061 11437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:16.208743 12066 leader_election.cc:290] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:16.208760 11437 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.207978 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5787304dfe4697baeaeb367ee7dda3" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:16.210446 11516 raft_consensus.cc:3055] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:17:16.210731 11516 raft_consensus.cc:740] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:16.211484 11516 consensus_queue.cc:260] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:16.214922 11516 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.216692 11437 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:16.208405 11366 raft_consensus.cc:3060] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:16.249861 11517 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:16.251579 17560 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:16.270366 11369 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:16.272007 17598 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:16.294786 11370 raft_consensus.cc:1275] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:17:16.302372 17571 consensus_queue.cc:1048] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:16.308379 11519 raft_consensus.cc:1275] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:16.331969 17612 consensus_queue.cc:1048] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:16.356909 11516 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:16.378290 12032 leader_election.cc:304] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:16.384270 17612 raft_consensus.cc:2804] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:16.398000 11437 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:16.399744 17471 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:16.385784 11366 raft_consensus.cc:2468] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:16.420751 17612 raft_consensus.cc:697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:16.460602 17612 consensus_queue.cc:237] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:16.510336 11369 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:16.528496 17641 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
W20260709 13:17:16.593323 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 54409.4us
W20260709 13:17:16.598523 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 49987.3us
I20260709 13:17:16.602001 11516 raft_consensus.cc:1275] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:17:16.647490 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:16.705987 11266 catalog_manager.cc:5697] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 16 to 17, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 17 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:16.680405 17638 consensus_queue.cc:1048] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 13:17:16.741977 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 75654.7us
W20260709 13:17:16.747582 11334 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 144552us
I20260709 13:17:16.755035 11516 raft_consensus.cc:1275] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:17:16.761010 17471 consensus_queue.cc:1048] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
W20260709 13:17:16.816411 11335 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 211565us
I20260709 13:17:16.903815 11369 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:16.906347 17471 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:16.930512 11437 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:16.931986 17647 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:16.988623 11516 raft_consensus.cc:1275] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:17:16.990926 17647 consensus_queue.cc:1048] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:17.009879 11437 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:17:17.049489 11369 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:17.051429 17660 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:17:17.097761 11409 consensus_peers.cc:597] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 -> Peer b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): Couldn't send request to peer b1906ab37ba541f493936685a808c8a9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81ba42e5875044cb8ebec6e017d5585b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:17.104539 11516 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:17:17.105994 17655 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:17:17.130091 17660 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:17.132745 11366 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:17.161053 11262 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New cstate: current_term: 13 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:17.209873 11517 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER b1906ab37ba541f493936685a808c8a9 (127.10.4.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } } }
I20260709 13:17:17.506022 17672 ts_tablet_manager.cc:933] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: Initiating tablet copy from peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:17.536955 17672 tablet_copy_client.cc:323] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.4.66:44257
I20260709 13:17:17.542150 11455 tablet_copy_service.cc:140] P 6e87b213a4554d94825aceec352fc094: Received BeginTabletCopySession request for tablet 81ba42e5875044cb8ebec6e017d5585b from peer b1906ab37ba541f493936685a808c8a9 ({username='slave'} at 127.0.0.1:41514)
I20260709 13:17:17.542817 11455 tablet_copy_service.cc:161] P 6e87b213a4554d94825aceec352fc094: Beginning new tablet copy session on tablet 81ba42e5875044cb8ebec6e017d5585b from peer b1906ab37ba541f493936685a808c8a9 at {username='slave'} at 127.0.0.1:41514: session id = b1906ab37ba541f493936685a808c8a9-81ba42e5875044cb8ebec6e017d5585b
I20260709 13:17:17.551543 11455 tablet_copy_source_session.cc:215] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:17.570654 17672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ba42e5875044cb8ebec6e017d5585b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:17.659507 17672 tablet_copy_client.cc:806] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:17.660149 17672 tablet_copy_client.cc:670] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:17.667719 17672 tablet_copy_client.cc:538] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:17.704792 17672 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: Bootstrap starting.
I20260709 13:17:17.742274 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:17.809464 17672 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:17.810580 17672 tablet_bootstrap.cc:492] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: Bootstrap complete.
I20260709 13:17:17.811349 17672 ts_tablet_manager.cc:1403] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: Time spent bootstrapping tablet: real 0.107s	user 0.031s	sys 0.009s
I20260709 13:17:17.832947 17672 raft_consensus.cc:359] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:17:17.841508 17672 raft_consensus.cc:740] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Initialized, Role: LEARNER
I20260709 13:17:17.847916 17672 consensus_queue.cc:260] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: NON_VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: true } }
I20260709 13:17:17.857455 17672 ts_tablet_manager.cc:1434] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 13:17:17.882972 11455 tablet_copy_service.cc:342] P 6e87b213a4554d94825aceec352fc094: Request end of tablet copy session b1906ab37ba541f493936685a808c8a9-81ba42e5875044cb8ebec6e017d5585b received from {username='slave'} at 127.0.0.1:41514
I20260709 13:17:17.883620 11455 tablet_copy_service.cc:434] P 6e87b213a4554d94825aceec352fc094: ending tablet copy session b1906ab37ba541f493936685a808c8a9-81ba42e5875044cb8ebec6e017d5585b on tablet 81ba42e5875044cb8ebec6e017d5585b with peer b1906ab37ba541f493936685a808c8a9
I20260709 13:17:17.895793 12065 raft_consensus.cc:1217] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:17:18.160385 11517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:38026
I20260709 13:17:18.161069 11517 raft_consensus.cc:606] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Received request to transfer leadership to TS 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:18.173249 11517 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284"
dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
mode: GRACEFUL
new_leader_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:38026
I20260709 13:17:18.173872 11517 raft_consensus.cc:606] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Received request to transfer leadership to TS 6e87b213a4554d94825aceec352fc094
I20260709 13:17:18.175173 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:18.176942 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:18.256768 17640 raft_consensus.cc:993] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:17:18.257315 17640 raft_consensus.cc:1081] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Signalling peer 01ae50c877d04ba687e4e06259ba5581 to start an election
I20260709 13:17:18.268467 11369 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7"
dest_uuid: "01ae50c877d04ba687e4e06259ba5581"
 from {username='slave'} at 127.0.0.1:37086
I20260709 13:17:18.269079 11369 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.269411 11369 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.283262 11369 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.286438 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:18.286988 12065 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.286455 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "01ae50c877d04ba687e4e06259ba5581" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:18.288297 11517 raft_consensus.cc:3055] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:17:18.288573 11517 raft_consensus.cc:740] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:18.289593 11517 consensus_queue.cc:260] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.290581 11517 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:18.293463 12065 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:18.294945 11334 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:18.295717 17649 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:18.296196 17649 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Becoming Leader. State: Replica: 01ae50c877d04ba687e4e06259ba5581, State: Running, Role: LEADER
I20260709 13:17:18.297000 17649 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.297516 11517 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01ae50c877d04ba687e4e06259ba5581 in term 16.
I20260709 13:17:18.293251 11369 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [CANDIDATE]: Term 16 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:18.312808 17660 raft_consensus.cc:1064] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: attempting to promote NON_VOTER b1906ab37ba541f493936685a808c8a9 to VOTER
I20260709 13:17:18.316159 17660 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.339177 12065 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 LEARNER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:18.340260 11517 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:18.340662 17663 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:18.341864 17560 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:18.353410 11262 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 reported cstate change: term changed from 15 to 16, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65). New cstate: current_term: 16 leader_uuid: "01ae50c877d04ba687e4e06259ba5581" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.380563 11369 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:17:18.386786 17660 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:18.420701 12065 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:18.414376 17560 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:18.455158 11268 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 reported cstate change: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:18.460110 11369 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:18.480931 11519 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, b1906ab37ba541f493936685a808c8a9 (127.10.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } attrs { replace: true } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:18.589087 17654 raft_consensus.cc:993] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: : Instructing follower 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:18.589556 17654 raft_consensus.cc:1081] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Signalling peer 6e87b213a4554d94825aceec352fc094 to start an election
I20260709 13:17:18.591197 11437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284"
dest_uuid: "6e87b213a4554d94825aceec352fc094"
 from {username='slave'} at 127.0.0.1:51474
I20260709 13:17:18.591817 11437 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.592145 11437 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:18.591128 11442 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.604236 11437 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.622439 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:18.623018 12066 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:18.624213 11437 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:18.630653 12066 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:18.632107 11409 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:18.632879 17683 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:18.633988 17683 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:18.636225 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:18.636782 11519 raft_consensus.cc:3055] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:18.637058 11519 raft_consensus.cc:740] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:18.637825 11519 consensus_queue.cc:260] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.634930 17683 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:18.646569 11519 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:18.683769 11264 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 12, leader changed from 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.695951 11517 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:18.697607 17560 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:18.705124 17663 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:18.709817 11517 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:18.746886 11254 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81ba42e5875044cb8ebec6e017d5585b with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:18.762521 11266 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 reported cstate change: config changed from index 11 to 12, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New cstate: current_term: 13 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.780457 11353 tablet_service.cc:1558] Processing DeleteTablet for tablet 81ba42e5875044cb8ebec6e017d5585b with delete_type TABLET_DATA_TOMBSTONED (TS 01ae50c877d04ba687e4e06259ba5581 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:34564
I20260709 13:17:18.807536 11519 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:18.818883 17689 tablet_replica.cc:333] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:18.832024 12066 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:18.835834 17689 raft_consensus.cc:2243] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.837729 17689 raft_consensus.cc:2272] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.842150 17689 ts_tablet_manager.cc:1916] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:18.859504 12066 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:18.874078 17649 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:18.906180 17689 ts_tablet_manager.cc:1929] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.11
I20260709 13:17:18.906759 17689 log.cc:1199] T 81ba42e5875044cb8ebec6e017d5585b P 01ae50c877d04ba687e4e06259ba5581: Deleting WAL directory at /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/wals/81ba42e5875044cb8ebec6e017d5585b
I20260709 13:17:18.910198 11253 catalog_manager.cc:5028] TS 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): tablet 81ba42e5875044cb8ebec6e017d5585b (table test-workload [id=860deb7e587340d8a0da392b46ada4ae]) successfully deleted
I20260709 13:17:18.912736 11519 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Refusing update from remote peer 01ae50c877d04ba687e4e06259ba5581: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:17:18.914135 17649 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:18.924280 17683 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:19.062022 11519 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:17:19.063860 17683 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:19.131434 12065 raft_consensus.cc:2955] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } } }
I20260709 13:17:19.136817 12061 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:17:19.153077 17660 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:17:19.178383 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:19.300748 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
W20260709 13:17:19.341243 12033 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
W20260709 13:17:19.423457 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 82240.4us
W20260709 13:17:19.428313 11409 consensus_peers.cc:597] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.430174 12033 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.436697 12033 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.437631 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:19.439603 10257 tablet_replica.cc:333] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.440482 10257 raft_consensus.cc:2243] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:19.440653 11483 consensus_peers.cc:597] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.441524 11483 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.441692 10257 raft_consensus.cc:2272] T 8a142ac56caa42a08e0a838dc87ac3b3 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.444149 10257 tablet_replica.cc:333] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.444836 10257 raft_consensus.cc:2243] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.445341 10257 raft_consensus.cc:2272] T 4ad5a541019548a3b543619a92b420b1 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.447476 10257 tablet_replica.cc:333] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.448128 10257 raft_consensus.cc:2243] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.448695 10257 raft_consensus.cc:2272] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.457038 10257 tablet_replica.cc:333] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.457970 10257 raft_consensus.cc:2243] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.458652 10257 raft_consensus.cc:2272] T a38969de5c824718aa81b9622624bc51 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.470665 11409 consensus_peers.cc:597] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.476058 10257 tablet_replica.cc:333] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
W20260709 13:17:19.476625 11409 consensus_peers.cc:597] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.477159 10257 raft_consensus.cc:2243] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.478014 10257 raft_consensus.cc:2272] T bab171deb5e946c8b074a4c091542a78 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.480832 10257 tablet_replica.cc:333] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.481642 10257 raft_consensus.cc:2243] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.482362 10257 raft_consensus.cc:2272] T 7a5787304dfe4697baeaeb367ee7dda3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.501595 10257 tablet_replica.cc:333] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.504541 10257 raft_consensus.cc:2243] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.506253 10257 raft_consensus.cc:2272] T 6f3368fb0be64c3b92da009af48a1854 P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.508733 10257 tablet_replica.cc:333] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.509529 10257 raft_consensus.cc:2243] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:19.510608 10257 raft_consensus.cc:2272] T 683aea8d9bd64288b5932ad265b60dd4 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.518636 11483 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.523326 10257 tablet_replica.cc:333] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.524331 10257 raft_consensus.cc:2243] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.524981 10257 raft_consensus.cc:2272] T 857937fa61dc400a81345944f0a14511 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.535135 11483 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.535048 12033 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.538058 12033 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.542541 11483 consensus_peers.cc:597] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.545982 10257 tablet_replica.cc:333] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
W20260709 13:17:19.548447 11409 consensus_peers.cc:597] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.556262 11409 consensus_peers.cc:597] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.562268 10257 raft_consensus.cc:2243] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.564695 10257 raft_consensus.cc:2272] T 116f1226682347599c239ad3bee95b41 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.567727 10257 tablet_replica.cc:333] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.568804 10257 raft_consensus.cc:2243] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.569478 10257 raft_consensus.cc:2272] T 99667fcf16014dc1b9b174f0a37d16af P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.581358 10257 tablet_replica.cc:333] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.583225 10257 raft_consensus.cc:2243] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:17:19.585485 10257 raft_consensus.cc:2272] T 7074ddeb0b7b4c1395b798352ff7db0a P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.601263 10257 tablet_replica.cc:333] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.602569 10257 raft_consensus.cc:2243] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:17:19.604970 10257 raft_consensus.cc:2272] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.606550 11483 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.607796 10257 tablet_replica.cc:333] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.608819 10257 raft_consensus.cc:2243] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.610602 10257 raft_consensus.cc:2272] T ab2785abe6bd4898ac2dbb25f53e4c8b P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.613178 11483 consensus_peers.cc:597] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.617049 10257 tablet_replica.cc:333] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.617908 10257 raft_consensus.cc:2243] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.618566 10257 raft_consensus.cc:2272] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.624782 10257 tablet_replica.cc:333] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.625583 10257 raft_consensus.cc:2243] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.626192 10257 raft_consensus.cc:2272] T bce024e6afd54cf0992c0af40df6a4eb P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.628749 10257 tablet_replica.cc:333] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.629490 10257 raft_consensus.cc:2243] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:19.630558 10257 raft_consensus.cc:2272] T 8b949bd9634b40af97ef4fb06ee21e9f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.633380 10257 tablet_replica.cc:333] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.634092 10257 raft_consensus.cc:2243] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:19.635114 10257 raft_consensus.cc:2272] T ae095c448d6e4a3ab9eaecd9695887a7 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.643887 11409 consensus_peers.cc:597] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.647579 10257 tablet_replica.cc:333] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.648576 10257 raft_consensus.cc:2243] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:19.654054 11483 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.649528 10257 raft_consensus.cc:2272] T b8ec7d9e61ab4ff5bd59a37021d68dba P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.662725 12033 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.670591 11409 consensus_peers.cc:597] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.679421 12033 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.696844 12033 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.710021 10257 tablet_replica.cc:333] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.710903 10257 raft_consensus.cc:2243] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.711519 10257 raft_consensus.cc:2272] T 9c1dd88ce9a04ff59ee55a115686f148 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.717844 12033 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.718434 11409 consensus_peers.cc:597] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.726859 11409 consensus_peers.cc:597] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.737138 10257 tablet_replica.cc:333] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.740110 10257 raft_consensus.cc:2243] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:17:19.743629 11483 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.748622 10257 raft_consensus.cc:2272] T 435f2e223e784d188897ac4adbc34211 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.756309 10257 tablet_replica.cc:333] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.757205 10257 raft_consensus.cc:2243] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.757850 10257 raft_consensus.cc:2272] T 114b7c3bbf65457b98b17b61f3c64839 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.761428 10257 tablet_replica.cc:333] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
W20260709 13:17:19.764114 11483 consensus_peers.cc:597] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.768112 10257 raft_consensus.cc:2243] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.768821 10257 raft_consensus.cc:2272] T bcf62f83a1754dcc80291daa521a9c06 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.774703 10257 tablet_replica.cc:333] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.801599 10257 raft_consensus.cc:2243] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:19.802768 10257 raft_consensus.cc:2272] T 1073b5e444bf4b34ad3dca3ea96efc76 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.809849 10257 tablet_replica.cc:333] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.816039 10257 raft_consensus.cc:2243] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.823791 10257 raft_consensus.cc:2272] T 8dd69c97ea4640a0a71117011ab4e5d4 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.826754 10257 tablet_replica.cc:333] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
W20260709 13:17:19.826876 11483 consensus_peers.cc:597] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.828790 10257 raft_consensus.cc:2243] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.831887 10257 raft_consensus.cc:2272] T 81760accce3a440a82323bd3ff53c551 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:19.833663 11409 consensus_peers.cc:597] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:19.863196 11409 consensus_peers.cc:597] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:19.870678 10257 tablet_replica.cc:333] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.878765 10257 raft_consensus.cc:2243] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.884624 10257 raft_consensus.cc:2272] T 38ecdb2d3e8745889c0343749502c9f5 P 01ae50c877d04ba687e4e06259ba5581 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.889642 10257 tablet_replica.cc:333] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.898578 10257 raft_consensus.cc:2243] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:17:19.901816 10257 raft_consensus.cc:2272] T df8439ad70bf422daa65806caf52f7f4 P 01ae50c877d04ba687e4e06259ba5581 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.913770 10257 tablet_replica.cc:333] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.914700 10257 raft_consensus.cc:2243] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.923343 10257 raft_consensus.cc:2272] T dbbdba17cce24fe7923cf755be2d3e0a P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.932324 10257 tablet_replica.cc:333] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.934157 10257 raft_consensus.cc:2243] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.946655 10257 raft_consensus.cc:2272] T e4aa2e377a0f45378e0df4a788c07ddf P 01ae50c877d04ba687e4e06259ba5581 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.949438 10257 tablet_replica.cc:333] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.955965 10257 raft_consensus.cc:2243] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:17:19.957538 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 99953us
I20260709 13:17:19.960184 10257 raft_consensus.cc:2272] T b01b3301ee794d99bcb4feb672ac699f P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.966931 10257 tablet_replica.cc:333] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.982635 10257 raft_consensus.cc:2243] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:19.987201 10257 raft_consensus.cc:2272] T 988262f8a4a846be8b350d9d99c4a730 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:19.990096 10257 tablet_replica.cc:333] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:19.999904 10257 raft_consensus.cc:2243] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:20.000787 10257 raft_consensus.cc:2272] T 7d370e878a484e1aaaa0df1466da6f24 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.016628 10257 tablet_replica.cc:333] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.017477 10257 raft_consensus.cc:2243] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:17:20.019827 10257 raft_consensus.cc:2272] T 78b9049e00f94cc1912970768037cf24 P 01ae50c877d04ba687e4e06259ba5581 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.036895 10257 tablet_replica.cc:333] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.037708 10257 raft_consensus.cc:2243] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:20.045837 10257 raft_consensus.cc:2272] T 3cd0c27e51bd41b5959c50a56353166e P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.048686 10257 tablet_replica.cc:333] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.056624 10257 raft_consensus.cc:2243] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:20.057353 10257 raft_consensus.cc:2272] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.070801 10257 tablet_replica.cc:333] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.082894 10257 raft_consensus.cc:2243] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:20.083669 10257 raft_consensus.cc:2272] T 44cbd38458af4b03b0d3f362bd6a2f85 P 01ae50c877d04ba687e4e06259ba5581 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.086400 10257 tablet_replica.cc:333] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.091945 10257 raft_consensus.cc:2243] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:20.092602 10257 raft_consensus.cc:2272] T 063871012da74857a239cdf01c76e5ab P 01ae50c877d04ba687e4e06259ba5581 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.095000 10257 tablet_replica.cc:333] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.103708 10257 raft_consensus.cc:2243] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:20.107137 10257 raft_consensus.cc:2272] T 2eaec0f849964172b0dcfba0af0e7fa3 P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.109767 10257 tablet_replica.cc:333] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.113459 10257 raft_consensus.cc:2243] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:20.121986 10257 raft_consensus.cc:2272] T ad3ab48ae4bf4452a19c99c71fd014f1 P 01ae50c877d04ba687e4e06259ba5581 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.128343 10257 tablet_replica.cc:333] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.129139 10257 raft_consensus.cc:2243] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:17:20.130131 10257 raft_consensus.cc:2272] T 593d6a54177d461680000f43998f8775 P 01ae50c877d04ba687e4e06259ba5581 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.137907 10257 tablet_replica.cc:333] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.138788 10257 raft_consensus.cc:2243] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:20.139865 10257 raft_consensus.cc:2272] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.142652 10257 tablet_replica.cc:333] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.151451 10257 raft_consensus.cc:2243] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:20.152635 10257 raft_consensus.cc:2272] T a7effda9a8a1409cbe25d55ed7cf2964 P 01ae50c877d04ba687e4e06259ba5581 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.155450 10257 tablet_replica.cc:333] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.156173 10257 raft_consensus.cc:2243] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:20.159363 10257 raft_consensus.cc:2272] T 741facf7b8264a10bc5c3431415fc34a P 01ae50c877d04ba687e4e06259ba5581 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.167328 10257 tablet_replica.cc:333] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581: stopping tablet replica
I20260709 13:17:20.168102 10257 raft_consensus.cc:2243] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:20.174134 10257 raft_consensus.cc:2272] T 3d475300e0054608a5d243d859d6609b P 01ae50c877d04ba687e4e06259ba5581 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:20.596773 17706 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.597354 17706 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:20.599807 17706 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:20.602001 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:20.609150 11483 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.637760 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:20.638680 17642 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:20.706454 17642 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.707046 17642 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:20.709450 17642 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:20.711412 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:17:20.719303 11483 leader_election.cc:336] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.719870 11483 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:20.722571 17683 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.723212 17683 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:20.727684 17683 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:20.728308 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:17:20.730028 11409 leader_election.cc:336] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.730926 11409 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:20.732211 17560 raft_consensus.cc:2749] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:20.733031 17642 raft_consensus.cc:2749] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:20.740931 17643 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.741521 17643 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:20.749162 12033 leader_election.cc:336] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.750272 17643 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:20.751732 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:20.752391 11441 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:20.753780 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:20.755227 17643 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:20.755708 17643 raft_consensus.cc:493] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.756162 17643 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:20.760407 17711 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.761068 17711 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:20.765537 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:20.763195 17711 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:20.773821 12033 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.774338 12033 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:20.775120 17711 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:20.794010 17643 raft_consensus.cc:515] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:20.801070 12033 leader_election.cc:336] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.803438 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b949bd9634b40af97ef4fb06ee21e9f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:20.804021 11441 raft_consensus.cc:3060] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:20.820097 11441 raft_consensus.cc:2468] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:20.821516 12034 leader_election.cc:304] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:20.822466 17711 raft_consensus.cc:2804] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:20.822970 17711 raft_consensus.cc:697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:20.824570 17683 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.825099 17683 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:20.823854 17711 consensus_queue.cc:237] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:20.830636 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:20.832437 17683 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:20.841053 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:20.842286 17643 leader_election.cc:290] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:20.846787 11409 leader_election.cc:336] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.847307 11409 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:20.851642 17560 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:20.866828 17642 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.867509 17642 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:20.869803 17642 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:20.872862 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:20.890530 11264 catalog_manager.cc:5697] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 16 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 17 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:20.914934 11483 leader_election.cc:336] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.915721 11483 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:20.918820 17642 raft_consensus.cc:2749] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:20.964229 17660 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.964881 17660 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:20.966998 17660 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:20.968884 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:20.969587 11517 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 16.
I20260709 13:17:20.972765 11408 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:20.973937 17683 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:20.974318 17683 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.974643 17683 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:17:20.981702 11409 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.982179 17683 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:20.988952 17706 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.989547 17706 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:20.991643 17706 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:20.993283 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
W20260709 13:17:20.996505 11483 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.997013 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:20.997181 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "683aea8d9bd64288b5932ad265b60dd4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:20.997816 11517 raft_consensus.cc:3060] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:20.998337 17706 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:20.999060 17706 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:20.997843 17718 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.000178 17718 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.001185 11409 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:20.999553 17706 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.004932 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:21.005012 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.005578 11441 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:17:21.005684 11516 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:21.007287 17706 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.007958 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.008893 17696 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:21.009306 17696 raft_consensus.cc:493] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.009650 17696 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:17:21.010051 12033 leader_election.cc:336] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.011569 17660 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.012323 17660 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.014384 17660 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.016589 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:21.002286 17718 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:21.018064 11483 leader_election.cc:336] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.019178 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:21.019850 12066 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 13.
I20260709 13:17:21.021453 11517 raft_consensus.cc:2468] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 17.
I20260709 13:17:21.022902 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.025187 17660 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:21.025223 11408 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.025918 17660 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.026458 17560 raft_consensus.cc:2804] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:21.026407 17660 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:21.029466 17683 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.023658 17696 raft_consensus.cc:515] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.025225 11409 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.033234 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae095c448d6e4a3ab9eaecd9695887a7" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:21.033829 11517 raft_consensus.cc:3060] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:21.034652 11483 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:21.036199 17642 raft_consensus.cc:2749] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:21.037443 17696 leader_election.cc:290] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:21.039302 12033 leader_election.cc:336] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.042196 17718 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.042889 17718 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.046227 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.048097 17718 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.056885 11517 raft_consensus.cc:2468] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:21.057770 17718 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.058509 17718 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.059634 12033 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.060150 12033 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:21.060890 17696 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:21.063478 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:21.064106 11441 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:17:21.065867 17718 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.066838 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.067490 11516 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 12.
I20260709 13:17:21.068681 12032 leader_election.cc:304] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.069418 17718 raft_consensus.cc:2804] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:21.069804 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.070621 17696 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:21.071005 17696 raft_consensus.cc:493] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.071360 17696 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:21.075611 17718 raft_consensus.cc:697] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:21.079839 17718 consensus_queue.cc:237] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.085209 17696 raft_consensus.cc:515] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.088913 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:21.089453 11441 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:21.096894 11441 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:21.098971 17643 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.103610 17643 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.105396 17643 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.105993 17683 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.106515 17683 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.107769 17643 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.108309 17643 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.117223 12033 leader_election.cc:336] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.117833 17643 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.121249 17683 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.117381 17560 raft_consensus.cc:697] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 17 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:21.123301 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.123992 11516 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 11.
I20260709 13:17:21.125340 17560 consensus_queue.cc:237] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.127024 17643 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.127071 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:21.127696 17643 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.134418 11409 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.130419 17683 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:21.137588 17683 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.137955 17683 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:21.144789 17643 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.118219 17660 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.154845 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:21.155468 11409 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.155512 11516 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 18.
W20260709 13:17:21.156397 12033 leader_election.cc:336] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:21.156912 12033 leader_election.cc:336] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.157949 17643 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.126816 17696 leader_election.cc:290] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.159713 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.158407 17643 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.160661 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c0e900e01b4e5aa4cf2fa0acbf6e86" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:21.164237 11517 raft_consensus.cc:3060] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:21.164103 11264 catalog_manager.cc:5697] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 16 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 17 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.170231 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.172137 17706 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.172647 17706 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.174683 17706 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.163153 17660 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.178737 11517 raft_consensus.cc:2468] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 13.
I20260709 13:17:21.178620 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435f2e223e784d188897ac4adbc34211" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:21.178654 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.179441 12066 raft_consensus.cc:3060] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:21.175164 17706 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.180572 17643 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.187376 12066 raft_consensus.cc:2468] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 14.
I20260709 13:17:21.208724 11262 catalog_manager.cc:5697] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 16 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 17 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.188028 17724 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:21.211975 17724 raft_consensus.cc:493] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.212575 17724 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:17:21.213105 12033 leader_election.cc:336] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.213541 12033 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:21.219856 17643 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:21.220906 12033 leader_election.cc:336] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.221138 17716 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.226171 17716 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.190659 12032 leader_election.cc:304] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.228190 17716 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:21.225442 12033 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:21.229928 11409 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.230182 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:21.230830 12066 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
W20260709 13:17:21.187593 11483 leader_election.cc:336] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.231007 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:21.231660 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 44171.8us
I20260709 13:17:21.231860 11517 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:21.232641 17718 raft_consensus.cc:2804] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:21.235517 12032 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.240365 17725 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:21.193316 17706 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.241359 11441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:21.241638 17723 raft_consensus.cc:2804] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:21.245918 17660 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.246414 17660 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.247799 17723 raft_consensus.cc:697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:21.248683 17723 consensus_queue.cc:237] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.253459 17643 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.253993 17643 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.220528 17724 raft_consensus.cc:515] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.262606 11408 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 92658.7us
I20260709 13:17:21.265213 17643 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.269641 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.273051 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.273871 17642 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:21.274258 17642 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.274603 17642 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:21.189406 17683 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.274173 17711 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:21.288076 17660 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.289923 17642 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.293047 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:17:21.295862 11409 leader_election.cc:336] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.297508 11409 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:21.304308 17642 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.306423 17728 raft_consensus.cc:2749] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:21.307356 17718 raft_consensus.cc:697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:21.308537 17683 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.308430 17718 consensus_queue.cc:237] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.310038 17724 leader_election.cc:290] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.280920 17711 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.311345 17711 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:21.287242 17706 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.313704 17706 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:21.315620 12034 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 79515.8us
I20260709 13:17:21.318501 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7074ddeb0b7b4c1395b798352ff7db0a" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:21.319130 11517 raft_consensus.cc:3060] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:21.325953 11517 raft_consensus.cc:2468] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 19.
I20260709 13:17:21.334801 12032 leader_election.cc:304] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.336011 17736 raft_consensus.cc:2804] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:21.314250 17706 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.338116 17706 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.362390 11262 catalog_manager.cc:5697] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 13 to 14, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 14 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.343477 17736 raft_consensus.cc:697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 19 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:21.368398 17736 consensus_queue.cc:237] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.379453 17711 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.381217 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 103568us
W20260709 13:17:21.394723 12033 leader_election.cc:336] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.398422 11442 raft_consensus.cc:1275] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:17:21.401225 11409 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.400564 11262 catalog_manager.cc:5697] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 12 to 13, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 13 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.407819 17711 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.410195 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:21.411473 11516 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b1906ab37ba541f493936685a808c8a9 in current term 18: Already voted for candidate 34a42aa6aa0a4778a532c6433d70b981 in this term.
W20260709 13:17:21.413053 10257 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11391)
I20260709 13:17:21.430058 17711 consensus_queue.cc:1048] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:17:21.430505 12033 leader_election.cc:336] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.431005 12033 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:21.433087 12033 leader_election.cc:336] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:21.433708 12033 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:21.438594 17643 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:17:21.444108 12033 leader_election.cc:336] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.444998 12033 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:21.444495 11264 catalog_manager.cc:5697] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 18 to 19, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 19 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:21.447023 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 172602us
W20260709 13:17:21.457998 11483 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.458978 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:21.461233 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df8439ad70bf422daa65806caf52f7f4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:21.462108 11516 raft_consensus.cc:3060] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:21.463546 17706 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:21.468890 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:21.469159 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:21.469568 12066 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:17:21.471899 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:21.472457 12066 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:21.446369 17718 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:17:21.475045 12065 raft_consensus.cc:2393] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 34a42aa6aa0a4778a532c6433d70b981 in current term 18: Already voted for candidate b1906ab37ba541f493936685a808c8a9 in this term.
I20260709 13:17:21.487998 17729 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.488629 17729 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.490739 17729 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.493877 11516 raft_consensus.cc:2468] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 12.
I20260709 13:17:21.499017 11408 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.501823 17729 raft_consensus.cc:2804] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:21.502506 17729 raft_consensus.cc:697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:21.505357 17729 consensus_queue.cc:237] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.525066 17740 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.525741 17740 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.528903 17740 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.530997 11517 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.532167 11517 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 20.
I20260709 13:17:21.535392 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:21.536092 17738 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
W20260709 13:17:21.536254 11409 leader_election.cc:336] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.536892 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.544426 17738 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.545758 17732 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:17:21.546278 17732 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.546705 17732 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:17:21.544004 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:21.549821 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.554116 11408 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e87b213a4554d94825aceec352fc094; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:21.555521 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:21.556262 11442 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 15.
I20260709 13:17:21.555590 17683 raft_consensus.cc:2749] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:21.565508 11409 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.557749 17731 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:21.567730 17731 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.565748 11264 catalog_manager.cc:5697] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 11 to 12, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 12 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.554891 17706 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:21.568686 17706 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.569056 17706 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:17:21.571156 11483 leader_election.cc:336] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:21.571975 11483 leader_election.cc:336] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.572359 11483 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:21.561291 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:21.576355 11483 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.546904 17738 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:21.580205 17734 raft_consensus.cc:2749] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:17:21.568120 17731 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:21.582458 12033 leader_election.cc:336] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.590497 17706 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.594858 17734 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.595532 17734 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.597580 17734 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:21.599305 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094" is_pre_election: true
I20260709 13:17:21.599939 11442 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 15.
I20260709 13:17:21.601183 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:21.602073 17734 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:21.602504 17734 raft_consensus.cc:493] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.602898 17734 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:21.604781 17737 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:17:21.605212 17737 raft_consensus.cc:493] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.605541 17737 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:17:21.610733 12033 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:21.617544 17706 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.620186 17732 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.622251 17732 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:21.627391 17730 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.628050 17730 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.629328 17742 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.630183 17730 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.629895 17742 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.631985 17742 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.636714 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b01b3301ee794d99bcb4feb672ac699f" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:21.637416 12065 raft_consensus.cc:3060] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:21.644306 17737 raft_consensus.cc:515] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.650440 12065 raft_consensus.cc:2468] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
W20260709 13:17:21.650710 12033 leader_election.cc:336] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.636317 17731 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.653203 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "593d6a54177d461680000f43998f8775" candidate_uuid: "6e87b213a4554d94825aceec352fc094" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:21.653445 17731 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.653829 11516 raft_consensus.cc:3060] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:17:21.655560 17737 leader_election.cc:290] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:21.656615 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d475300e0054608a5d243d859d6609b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:21.657182 11442 raft_consensus.cc:3060] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:21.660455 11516 raft_consensus.cc:2468] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e87b213a4554d94825aceec352fc094 in term 21.
W20260709 13:17:21.662695 11483 leader_election.cc:336] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.672683 11442 raft_consensus.cc:2468] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
I20260709 13:17:21.674073 12034 leader_election.cc:304] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:21.676463 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:21.677135 12065 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:21.677832 12061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:21.678481 12061 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:21.677767 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b9049e00f94cc1912970768037cf24" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:21.679068 12066 raft_consensus.cc:3060] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:21.682332 17718 raft_consensus.cc:2804] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:21.682893 17718 raft_consensus.cc:697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:21.683784 17718 consensus_queue.cc:237] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.684653 11408 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:21.685379 17732 raft_consensus.cc:2804] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:17:21.685834 17732 raft_consensus.cc:697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 21 LEADER]: Becoming Leader. State: Replica: 6e87b213a4554d94825aceec352fc094, State: Running, Role: LEADER
I20260709 13:17:21.686614 17732 consensus_queue.cc:237] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:21.689338 11483 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.693217 11517 raft_consensus.cc:1275] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:21.700178 17737 consensus_queue.cc:1048] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:17:21.706254 11483 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
W20260709 13:17:21.707412 11409 consensus_peers.cc:597] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:21.708108 17734 raft_consensus.cc:515] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.698578 12066 raft_consensus.cc:2468] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
W20260709 13:17:21.708112 11409 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.710242 17706 raft_consensus.cc:2804] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:21.710747 17706 raft_consensus.cc:697] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:21.711568 17706 consensus_queue.cc:237] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.712239 11442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b402b68ac7b4b9292a6ab6fb6fd3ea4" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6e87b213a4554d94825aceec352fc094"
I20260709 13:17:21.712828 11442 raft_consensus.cc:3060] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:17:21.715946 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.716647 11516 raft_consensus.cc:1275] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:17:21.718053 17723 consensus_queue.cc:1048] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:17:21.718158 17734 leader_election.cc:290] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:21.722693 17731 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:21.723121 17731 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.723493 17731 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:21.726169 11264 catalog_manager.cc:5697] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 reported cstate change: term changed from 20 to 21, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 6e87b213a4554d94825aceec352fc094 (127.10.4.66). New cstate: current_term: 21 leader_uuid: "6e87b213a4554d94825aceec352fc094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.727730 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.729091 17742 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:17:21.729439 17742 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 01ae50c877d04ba687e4e06259ba5581)
I20260709 13:17:21.729768 17742 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:21.730964 11442 raft_consensus.cc:2468] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 16.
I20260709 13:17:21.718120 11262 catalog_manager.cc:5697] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 15 to 16, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 16 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.737331 17742 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.738178 11409 consensus_peers.cc:597] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:21.739504 17742 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:21.741847 11484 leader_election.cc:304] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094; no voters: 
I20260709 13:17:21.744632 17741 raft_consensus.cc:2804] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:17:21.745174 17741 raft_consensus.cc:697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:21.746114 17741 consensus_queue.cc:237] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:21.745138 11483 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.765640 17734 raft_consensus.cc:2804] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:21.766228 17734 raft_consensus.cc:697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:21.767079 17734 consensus_queue.cc:237] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.768827 11483 leader_election.cc:336] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:21.773051 11483 leader_election.cc:336] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:21.773825 11483 leader_election.cc:336] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.779839 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "741facf7b8264a10bc5c3431415fc34a" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:21.780400 12066 raft_consensus.cc:3060] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:17:21.786990 12066 raft_consensus.cc:2468] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 18.
I20260709 13:17:21.779192 17731 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:21.793972 17731 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:21.809458 12033 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:21.820000 11262 catalog_manager.cc:5697] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 16 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 17 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.823381 12066 raft_consensus.cc:1275] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:17:21.825286 12066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7effda9a8a1409cbe25d55ed7cf2964" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:21.825853 12066 raft_consensus.cc:3060] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:21.829074 17732 consensus_queue.cc:1048] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:17:21.843678 12066 raft_consensus.cc:2468] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 17.
I20260709 13:17:21.829712 11483 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
W20260709 13:17:21.867102 11483 leader_election.cc:336] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.868690 17734 raft_consensus.cc:2804] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:17:21.867758 11483 leader_election.cc:336] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.856170 11262 catalog_manager.cc:5697] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 15 to 16, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 16 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.870287 11262 catalog_manager.cc:5697] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 12 to 13, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 13 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.871459 11483 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:21.873085 17734 raft_consensus.cc:697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:21.879650 11516 raft_consensus.cc:1275] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:17:21.874639 17741 raft_consensus.cc:2804] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:21.883584 17734 consensus_queue.cc:237] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.885038 11483 leader_election.cc:336] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:21.886082 17737 consensus_queue.cc:1048] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:17:21.885128 12033 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:21.881832 11409 consensus_peers.cc:597] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:21.891122 17741 raft_consensus.cc:697] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:21.892032 17741 consensus_queue.cc:237] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:21.895481 12033 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:21.907809 11409 consensus_peers.cc:597] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:21.917435 11517 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:21.926733 17738 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:17:21.932833 11409 consensus_peers.cc:597] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:21.970613 11483 consensus_peers.cc:597] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:21.973096 11409 consensus_peers.cc:597] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:21.983059 12033 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:21.995760 11441 raft_consensus.cc:1275] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:17:22.000020 17738 consensus_queue.cc:1048] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:17:22.012334 12033 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.014726 11264 catalog_manager.cc:5697] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 17 to 18, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 18 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:22.016647 11264 catalog_manager.cc:5697] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 16 to 17, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 17 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:22.023609 11483 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.024161 11483 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.032897 11409 consensus_peers.cc:597] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.056358 11409 consensus_peers.cc:597] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.064983 11483 consensus_peers.cc:597] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.087347 12033 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.104985 11483 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.110916 11409 consensus_peers.cc:597] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.111457 11409 consensus_peers.cc:597] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.114447 11516 raft_consensus.cc:1275] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:17:22.115566 17732 consensus_queue.cc:1048] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:17:22.129145 12033 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.135571 12065 raft_consensus.cc:1275] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:17:22.137692 17734 consensus_queue.cc:1048] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:17:22.245854 11409 consensus_peers.cc:597] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.254912 11442 raft_consensus.cc:1275] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:17:22.250200 12033 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.261404 17643 consensus_queue.cc:1048] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:17:22.283818 12033 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.286257 11409 consensus_peers.cc:597] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.283834 11484 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:44257, user_credentials={real_user=slave}} blocked reactor thread for 65120.7us
I20260709 13:17:22.299901 11516 raft_consensus.cc:1275] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Refusing update from remote peer 6e87b213a4554d94825aceec352fc094: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:17:22.312486 12061 raft_consensus.cc:1275] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:17:22.333671 17716 consensus_queue.cc:1048] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:22.346727 17770 consensus_queue.cc:1048] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:22.354401 12066 raft_consensus.cc:1275] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:17:22.352093 11409 consensus_peers.cc:597] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.357751 11483 consensus_peers.cc:597] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.358326 11483 consensus_peers.cc:597] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.358772 11483 consensus_peers.cc:597] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.359160 11483 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.359566 11483 consensus_peers.cc:597] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.359965 11483 consensus_peers.cc:597] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.362226 12033 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.394471 11409 consensus_peers.cc:597] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.396574 11409 consensus_peers.cc:597] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.453190 11442 raft_consensus.cc:1275] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:17:22.454433 17734 consensus_queue.cc:1048] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:17:22.453255 12033 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.468153 17716 consensus_queue.cc:1048] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:17:22.485140 12066 raft_consensus.cc:1275] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:17:22.489224 11483 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:22.491815 11483 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:22.645359 17768 consensus_queue.cc:1048] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:17:22.648032 17776 log.cc:927] Time spent T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 13:17:22.713429 11483 consensus_peers.cc:597] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.716678 11483 consensus_peers.cc:597] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.739187 11483 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:23.177240 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
I20260709 13:17:23.324746 17643 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.325481 17643 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:23.330096 17643 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:23.331683 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:23.334987 11519 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 18.
W20260709 13:17:23.334148 12033 leader_election.cc:336] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:23.339427 12032 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:23.344094 17777 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:23.344552 17777 raft_consensus.cc:493] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.344894 17777 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:23.361511 17777 raft_consensus.cc:515] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:23.371423 12033 leader_election.cc:336] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:23.373391 11519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1073b5e444bf4b34ad3dca3ea96efc76" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:23.374006 11519 raft_consensus.cc:3060] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:23.382159 11519 raft_consensus.cc:2468] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 19.
I20260709 13:17:23.383425 12032 leader_election.cc:304] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581
I20260709 13:17:23.384259 17643 raft_consensus.cc:2804] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:23.384398 17777 leader_election.cc:290] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 election: Requested vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:23.385030 17643 raft_consensus.cc:697] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 19 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:23.385918 17643 consensus_queue.cc:237] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:23.422174 11262 catalog_manager.cc:5697] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 17 to 19, leader changed from 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 19 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:23.424840 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
W20260709 13:17:23.458962 12034 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.525276 12034 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.547964 11484 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.568984 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:23.571981 10257 tablet_replica.cc:333] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.573212 10257 raft_consensus.cc:2243] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.574501 10257 raft_consensus.cc:2272] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.577519 10257 tablet_replica.cc:333] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.581981 10257 raft_consensus.cc:2243] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.583547 10257 raft_consensus.cc:2272] T 8a142ac56caa42a08e0a838dc87ac3b3 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.594066 10257 tablet_replica.cc:333] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.599424 10257 raft_consensus.cc:2243] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 15 LEADER]: Raft consensus shutting down.
W20260709 13:17:23.602802 11484 consensus_peers.cc:597] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.610708 10257 raft_consensus.cc:2272] T a38969de5c824718aa81b9622624bc51 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.613715 10257 tablet_replica.cc:333] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.614622 10257 raft_consensus.cc:2243] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.616052 10257 raft_consensus.cc:2272] T e6e7edf5f82e49158a4023ea61d78284 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.618834 10257 tablet_replica.cc:333] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
W20260709 13:17:23.623703 12034 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.624894 10257 raft_consensus.cc:2243] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.628973 10257 raft_consensus.cc:2272] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.631848 10257 tablet_replica.cc:333] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
W20260709 13:17:23.638217 12034 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.644516 10257 raft_consensus.cc:2243] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.645315 10257 raft_consensus.cc:2272] T 7a5787304dfe4697baeaeb367ee7dda3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.648940 10257 tablet_replica.cc:333] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.652120 10257 raft_consensus.cc:2243] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Raft consensus shutting down.
W20260709 13:17:23.654110 11484 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.656788 11484 consensus_peers.cc:597] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.657339 10257 raft_consensus.cc:2272] T b8ec7d9e61ab4ff5bd59a37021d68dba P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:23.660900 12034 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.665751 12034 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.666115 10257 tablet_replica.cc:333] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.666996 10257 raft_consensus.cc:2243] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.668399 10257 raft_consensus.cc:2272] T 9c1dd88ce9a04ff59ee55a115686f148 P 6e87b213a4554d94825aceec352fc094 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.680466 10257 tablet_replica.cc:333] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.681425 10257 raft_consensus.cc:2243] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:23.682727 11484 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.683908 10257 raft_consensus.cc:2272] T bab171deb5e946c8b074a4c091542a78 P 6e87b213a4554d94825aceec352fc094 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:23.687556 11484 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.687855 10257 tablet_replica.cc:333] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.694088 10257 raft_consensus.cc:2243] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.700191 10257 raft_consensus.cc:2272] T c753736d0d7f4578b8935b855a6b36e3 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.703058 10257 tablet_replica.cc:333] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
W20260709 13:17:23.705446 12034 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.717454 10257 raft_consensus.cc:2243] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.720443 10257 raft_consensus.cc:2272] T 6f3368fb0be64c3b92da009af48a1854 P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:23.724181 12034 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.733847 12033 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:23.734016 10257 tablet_replica.cc:333] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
W20260709 13:17:23.734607 11484 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.737408 11484 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.737977 12034 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.738461 10257 raft_consensus.cc:2243] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.739194 10257 raft_consensus.cc:2272] T 887116b7f3e447028e317f03386bf8d0 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.742098 10257 tablet_replica.cc:333] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.756124 10257 raft_consensus.cc:2243] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.757443 10257 raft_consensus.cc:2272] T 857937fa61dc400a81345944f0a14511 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:23.761327 12034 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.768220 10257 tablet_replica.cc:333] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.769232 10257 raft_consensus.cc:2243] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 17 LEADER]: Raft consensus shutting down.
W20260709 13:17:23.772812 12034 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.772886 10257 raft_consensus.cc:2272] T 683aea8d9bd64288b5932ad265b60dd4 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.778095 10257 tablet_replica.cc:333] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
W20260709 13:17:23.782732 12034 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.784318 10257 raft_consensus.cc:2243] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.789191 10257 raft_consensus.cc:2272] T 99667fcf16014dc1b9b174f0a37d16af P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.803655 10257 tablet_replica.cc:333] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.804557 10257 raft_consensus.cc:2243] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.805207 10257 raft_consensus.cc:2272] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.811615 10257 tablet_replica.cc:333] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.812394 10257 raft_consensus.cc:2243] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.813112 10257 raft_consensus.cc:2272] T 1ea18eac305644ed8d44618c2c0a8d0d P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.828523 10257 tablet_replica.cc:333] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.829324 10257 raft_consensus.cc:2243] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.830302 10257 raft_consensus.cc:2272] T bce024e6afd54cf0992c0af40df6a4eb P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:23.831141 12034 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.838562 10257 tablet_replica.cc:333] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
W20260709 13:17:23.842572 12033 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.844856 11514 raft_consensus.cc:1275] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:17:23.846570 17777 consensus_queue.cc:1048] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:23.851493 10257 raft_consensus.cc:2243] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.852524 10257 raft_consensus.cc:2272] T 8b949bd9634b40af97ef4fb06ee21e9f P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.857360 10257 tablet_replica.cc:333] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.858181 10257 raft_consensus.cc:2243] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.877671 10257 raft_consensus.cc:2272] T ab2785abe6bd4898ac2dbb25f53e4c8b P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.880663 10257 tablet_replica.cc:333] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.881484 10257 raft_consensus.cc:2243] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.882649 10257 raft_consensus.cc:2272] T e5b7e70b18df44b6b0e248fbc3664d96 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:23.896531 11484 consensus_peers.cc:597] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.901682 10257 tablet_replica.cc:333] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.902559 10257 raft_consensus.cc:2243] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.903164 10257 raft_consensus.cc:2272] T 7dab116931a64024ac61a35a6d29a836 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:23.917410 12034 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.918183 10257 tablet_replica.cc:333] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.919370 10257 raft_consensus.cc:2243] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.920418 10257 raft_consensus.cc:2272] T 572d79fba792428db7fcc1994fff81f7 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.923229 10257 tablet_replica.cc:333] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.926772 10257 raft_consensus.cc:2243] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.928012 10257 raft_consensus.cc:2272] T 81ba42e5875044cb8ebec6e017d5585b P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.930864 10257 tablet_replica.cc:333] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.931730 10257 raft_consensus.cc:2243] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.932374 10257 raft_consensus.cc:2272] T 98fc3c2677604f3bb787b67c5e5b6012 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.936645 10257 tablet_replica.cc:333] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.937439 10257 raft_consensus.cc:2243] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:23.938162 10257 raft_consensus.cc:2272] T 47922bb832834e72bc79e1e8159e6c1f P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.941105 10257 tablet_replica.cc:333] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.941874 10257 raft_consensus.cc:2243] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.943114 10257 raft_consensus.cc:2272] T 435f2e223e784d188897ac4adbc34211 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.965653 10257 tablet_replica.cc:333] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.966598 10257 raft_consensus.cc:2243] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:23.974781 10257 raft_consensus.cc:2272] T 114b7c3bbf65457b98b17b61f3c64839 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:23.978227 10257 tablet_replica.cc:333] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:23.979100 10257 raft_consensus.cc:2243] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:23.988878 12034 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:23.991103 11484 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:23.979739 10257 raft_consensus.cc:2272] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:24.000826 11484 consensus_peers.cc:597] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:24.001994 10257 tablet_replica.cc:333] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.002936 10257 raft_consensus.cc:2243] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.004070 10257 raft_consensus.cc:2272] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 6e87b213a4554d94825aceec352fc094 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.006636 10257 tablet_replica.cc:333] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.007423 10257 raft_consensus.cc:2243] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.008354 10257 raft_consensus.cc:2272] T 7d370e878a484e1aaaa0df1466da6f24 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.026929 10257 tablet_replica.cc:333] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.027808 10257 raft_consensus.cc:2243] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.028680 10257 raft_consensus.cc:2272] T 3cd0c27e51bd41b5959c50a56353166e P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.031441 10257 tablet_replica.cc:333] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.032421 10257 raft_consensus.cc:2243] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.038625 10257 raft_consensus.cc:2272] T 8dd69c97ea4640a0a71117011ab4e5d4 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.073218 10257 tablet_replica.cc:333] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.075384 10257 raft_consensus.cc:2243] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:24.076647 10257 raft_consensus.cc:2272] T 81760accce3a440a82323bd3ff53c551 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.083240 10257 tablet_replica.cc:333] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.084110 10257 raft_consensus.cc:2243] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:24.085491 10257 raft_consensus.cc:2272] T 38ecdb2d3e8745889c0343749502c9f5 P 6e87b213a4554d94825aceec352fc094 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.088551 10257 tablet_replica.cc:333] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.089730 10257 raft_consensus.cc:2243] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:17:24.095709 10257 raft_consensus.cc:2272] T df8439ad70bf422daa65806caf52f7f4 P 6e87b213a4554d94825aceec352fc094 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.098917 10257 tablet_replica.cc:333] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.099733 10257 raft_consensus.cc:2243] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.100488 10257 raft_consensus.cc:2272] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.103610 10257 tablet_replica.cc:333] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
W20260709 13:17:24.104200 12033 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:24.104353 10257 raft_consensus.cc:2243] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:24.105644 10257 raft_consensus.cc:2272] T 2eaec0f849964172b0dcfba0af0e7fa3 P 6e87b213a4554d94825aceec352fc094 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.119879 10257 tablet_replica.cc:333] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.120718 10257 raft_consensus.cc:2243] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:17:24.122238 10257 raft_consensus.cc:2272] T 593d6a54177d461680000f43998f8775 P 6e87b213a4554d94825aceec352fc094 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.125052 10257 tablet_replica.cc:333] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.126040 10257 raft_consensus.cc:2243] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.127004 10257 raft_consensus.cc:2272] T 4ad5a541019548a3b543619a92b420b1 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.132682 10257 tablet_replica.cc:333] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.133479 10257 raft_consensus.cc:2243] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.135954 10257 raft_consensus.cc:2272] T cf199b70fbb94d728a09194d18ca3b1e P 6e87b213a4554d94825aceec352fc094 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.138685 10257 tablet_replica.cc:333] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.139473 10257 raft_consensus.cc:2243] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.140283 10257 raft_consensus.cc:2272] T ad3ab48ae4bf4452a19c99c71fd014f1 P 6e87b213a4554d94825aceec352fc094 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.143239 10257 tablet_replica.cc:333] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.145646 10257 raft_consensus.cc:2243] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.153375 10257 raft_consensus.cc:2272] T 44cbd38458af4b03b0d3f362bd6a2f85 P 6e87b213a4554d94825aceec352fc094 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.156476 10257 tablet_replica.cc:333] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.157358 10257 raft_consensus.cc:2243] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.158041 10257 raft_consensus.cc:2272] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:24.160476 10257 tablet_replica.cc:333] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094: stopping tablet replica
I20260709 13:17:24.161134 10257 raft_consensus.cc:2243] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:24.161860 10257 raft_consensus.cc:2272] T 3d475300e0054608a5d243d859d6609b P 6e87b213a4554d94825aceec352fc094 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:24.225802 12033 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:24.244493 12033 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:24.351454 12034 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 450 similar messages]
W20260709 13:17:24.405170 12033 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.464954 11483 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.487913 12033 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.520177 12033 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.522460 12033 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:24.539002 11483 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.550820 11483 consensus_peers.cc:597] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.578022 12033 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.595630 11483 consensus_peers.cc:597] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.614398 11483 consensus_peers.cc:597] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:24.650648 17785 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.651288 17785 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:24.654202 17785 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:24.655776 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
W20260709 13:17:24.662876 11484 leader_election.cc:336] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:24.663319 11484 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:24.664008 17785 raft_consensus.cc:2749] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:24.673051 17777 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.673622 17777 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:24.675695 17777 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:24.676271 11483 consensus_peers.cc:597] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:24.685071 12034 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:24.685827 12033 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:24.686213 12033 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:24.687692 17789 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:24.725543 11483 consensus_peers.cc:597] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.745334 12033 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.745441 11483 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:24.751191 11483 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.754503 12033 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.771422 12033 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:24.783941 11483 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:24.791460 17768 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.791967 17768 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:24.794040 17768 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:24.800196 11483 leader_election.cc:336] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:24.801956 11484 leader_election.cc:336] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:24.802356 11484 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:24.803433 17768 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:17:24.804405 17643 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.804908 17643 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:24.806813 17643 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:24.808111 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:24.811020 12034 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:24.811442 12034 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:24.812099 17643 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:24.819245 17768 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.819813 17768 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:24.821753 17768 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:24.822960 17777 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.823475 17777 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:24.824018 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:24.824595 12065 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 12.
I20260709 13:17:24.825343 17777 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:24.825644 11483 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 
I20260709 13:17:24.827075 17768 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:24.827440 17768 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.827780 17768 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:24.828426 11484 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:24.833956 12033 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:24.835693 12034 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:24.836119 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:24.837155 17777 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:24.835381 17768 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:24.845382 17768 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:24.849151 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e7edf5f82e49158a4023ea61d78284" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:24.849704 12065 raft_consensus.cc:3060] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:17:24.851308 11484 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:24.856356 12065 raft_consensus.cc:2468] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:17:24.857829 11483 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:24.859202 17768 raft_consensus.cc:2804] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:17:24.860389 17768 raft_consensus.cc:697] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:24.861251 17768 consensus_queue.cc:237] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:24.903228 11483 consensus_peers.cc:597] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:24.907902 11262 catalog_manager.cc:5697] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 12 to 13, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 13 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:24.935640 11483 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:24.935186 17777 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:24.936476 17777 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:24.950549 12033 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:24.956471 12034 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:24.957011 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:24.957758 17643 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:24.961210 17777 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:24.965988 11483 consensus_peers.cc:597] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:25.032284 12033 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:25.038669 17643 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.039462 17643 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:25.042484 11483 consensus_peers.cc:597] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:25.044963 17768 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.045523 17768 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:25.049691 17768 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:25.049826 12033 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:25.053342 11484 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:25.057555 12034 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.058043 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.058692 17789 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.062461 17789 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.064755 17789 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:25.059234 17777 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:25.066501 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
W20260709 13:17:25.067034 11483 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.067198 11514 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 19.
I20260709 13:17:25.067505 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.058513 17643 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:25.068984 12034 leader_election.cc:336] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.070583 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.072008 17643 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:17:25.072373 17643 raft_consensus.cc:493] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.072669 17643 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:25.076397 17785 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:25.087212 17768 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.087833 17768 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.091051 17768 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:25.094787 11484 leader_election.cc:336] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.099129 12067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:25.099450 17789 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.099967 17789 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.102309 17789 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:25.111250 12033 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.130115 17777 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.130759 17777 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.133685 17777 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:25.136986 17789 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:17:25.137046 12033 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.137542 17789 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.140329 17789 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:25.144824 12033 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.159965 17777 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.160508 17777 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.169247 11483 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 6e87b213a4554d94825aceec352fc094, b1906ab37ba541f493936685a808c8a9
I20260709 13:17:25.171903 17768 raft_consensus.cc:2749] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:25.178409 17799 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.200932 17801 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.201624 17801 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:25.210000 11484 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.211336 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:25.179008 17799 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:25.217600 12034 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.218079 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.218873 17800 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:25.222500 11484 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.225683 17643 raft_consensus.cc:515] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.232414 17777 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:25.236263 12034 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.236785 12034 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:25.237689 12034 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.238080 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.238746 17800 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:25.238904 12034 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.239460 17800 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:25.239342 12034 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.248207 17777 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:25.254113 17801 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:25.254631 17799 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:25.259959 17800 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.260604 17800 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.263177 17800 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:25.260095 12034 leader_election.cc:336] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.268177 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a992bbbf8dd4ccbb70ff8d65e31edae" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:25.268738 11514 raft_consensus.cc:3060] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:25.269142 17799 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.269695 17799 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.271324 11516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981" is_pre_election: true
I20260709 13:17:25.273087 11516 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 16.
W20260709 13:17:25.278609 12034 leader_election.cc:336] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.278776 11514 raft_consensus.cc:2468] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 20.
I20260709 13:17:25.280076 17643 leader_election.cc:290] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:25.283252 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.284646 17800 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:17:25.285050 17800 raft_consensus.cc:493] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.285392 17800 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:25.287948 12032 leader_election.cc:304] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.289203 17789 raft_consensus.cc:2804] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:17:25.289681 17789 raft_consensus.cc:697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 20 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:25.290519 17789 consensus_queue.cc:237] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.299837 17768 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.300401 17768 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:25.302361 17768 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:25.317214 11262 catalog_manager.cc:5697] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 19 to 20, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 20 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:25.329759 11484 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.332885 17799 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
I20260709 13:17:25.361999 17800 raft_consensus.cc:515] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:25.368768 12034 leader_election.cc:336] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.370020 17799 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.370599 17799 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:25.372891 17799 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:25.374372 11484 consensus_peers.cc:597] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:25.376770 11484 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.386780 11514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5b7e70b18df44b6b0e248fbc3664d96" candidate_uuid: "b1906ab37ba541f493936685a808c8a9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "34a42aa6aa0a4778a532c6433d70b981"
I20260709 13:17:25.387503 11514 raft_consensus.cc:3060] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:17:25.389037 17800 leader_election.cc:290] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:25.391453 10257 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11466)
I20260709 13:17:25.394461 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9" is_pre_election: true
I20260709 13:17:25.417968 12065 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 13.
I20260709 13:17:25.394961 11514 raft_consensus.cc:2468] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1906ab37ba541f493936685a808c8a9 in term 17.
I20260709 13:17:25.419668 12032 leader_election.cc:304] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.420747 17800 raft_consensus.cc:2804] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:17:25.426194 17800 raft_consensus.cc:697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Becoming Leader. State: Replica: b1906ab37ba541f493936685a808c8a9, State: Running, Role: LEADER
I20260709 13:17:25.441829 17789 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.442420 17789 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.444830 17789 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:25.452114 12033 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:25.453224 12034 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.453624 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.454290 17643 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:25.449591 11484 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.456763 17800 consensus_queue.cc:237] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:25.417032 11483 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.461891 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.464253 17799 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:17:25.465065 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 48067.7us
W20260709 13:17:25.465517 11483 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.465859 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.470496 17785 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:25.472684 11483 consensus_peers.cc:597] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:25.473305 11483 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.473676 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.481789 17799 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:25.495154 11262 catalog_manager.cc:5697] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 reported cstate change: term changed from 16 to 17, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to b1906ab37ba541f493936685a808c8a9 (127.10.4.68). New cstate: current_term: 17 leader_uuid: "b1906ab37ba541f493936685a808c8a9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.511500 17785 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.512090 17785 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:25.514180 17785 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:25.534314 12067 raft_consensus.cc:1275] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 13:17:25.536087 11484 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.565632 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.570197 17799 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:17:25.570618 17799 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:25.570957 17799 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:25.581841 17799 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:25.610498 17768 consensus_queue.cc:1048] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:17:25.618884 17799 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Requested vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891)
W20260709 13:17:25.619650 11483 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:25.620232 11483 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.625236 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.628017 17768 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:25.635932 11484 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:25.651573 12065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ba42e5875044cb8ebec6e017d5585b" candidate_uuid: "34a42aa6aa0a4778a532c6433d70b981" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b1906ab37ba541f493936685a808c8a9"
I20260709 13:17:25.652223 12065 raft_consensus.cc:3060] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:17:25.658869 12065 raft_consensus.cc:2468] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a42aa6aa0a4778a532c6433d70b981 in term 14.
I20260709 13:17:25.663342 11483 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981, b1906ab37ba541f493936685a808c8a9; no voters: 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.664180 17799 raft_consensus.cc:2804] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:17:25.665040 17799 raft_consensus.cc:697] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Becoming Leader. State: Replica: 34a42aa6aa0a4778a532c6433d70b981, State: Running, Role: LEADER
I20260709 13:17:25.665975 17799 consensus_queue.cc:237] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:25.696415 11483 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:25.696887 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:25.697594 17768 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:17:25.725301 11262 catalog_manager.cc:5697] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 reported cstate change: term changed from 13 to 14, leader changed from 6e87b213a4554d94825aceec352fc094 (127.10.4.66) to 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67). New cstate: current_term: 14 leader_uuid: "34a42aa6aa0a4778a532c6433d70b981" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:25.818913 12034 consensus_peers.cc:597] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:25.840790 12034 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:25.876932 11514 raft_consensus.cc:1275] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:17:25.879745 17800 consensus_queue.cc:1048] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:17:25.898516 12034 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:25.969161 11514 raft_consensus.cc:1275] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Refusing update from remote peer b1906ab37ba541f493936685a808c8a9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:17:25.972409 17800 consensus_queue.cc:1048] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:17:25.993821 11484 consensus_peers.cc:597] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.005275 12034 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.020035 12034 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.032193 12034 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.043792 12034 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.071318 12033 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.133486 11484 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.149039 11484 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.163383 12034 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.163938 12034 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.167708 12034 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.172724 11484 consensus_peers.cc:597] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.193502 12034 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.195716 11484 consensus_peers.cc:597] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:26.225106 12067 raft_consensus.cc:1275] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Refusing update from remote peer 34a42aa6aa0a4778a532c6433d70b981: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:17:26.235965 11484 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:26.240145 17768 consensus_queue.cc:1048] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:17:26.254868 12033 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:26.257470 11484 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.285567 11484 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.306293 12034 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.336071 12034 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.395890 11484 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.417797 10257 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 11466)
W20260709 13:17:26.420261 12034 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.427584 11484 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.438925 12034 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:26.446952 17768 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:26.448804 17768 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:26.451116 17768 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:26.456107 11484 consensus_peers.cc:597] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:26.456609 11484 leader_election.cc:336] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:26.485450 11483 leader_election.cc:336] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:26.486222 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:26.487104 17768 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:26.501884 12033 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:26.515193 11484 consensus_peers.cc:597] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:26.571908 17643 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:26.572409 17643 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:26.574404 17643 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:26.580628 12033 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:26.582594 12034 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:26.583033 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:26.584489 17643 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:26.598848 12034 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:26.635372 17800 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:26.635931 17800 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:26.643568 12033 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:26.649276 12034 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:26.649714 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:26.650494 17643 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:26.651365 17800 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:26.687343 12034 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:26.719786 17800 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:26.732342 17800 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:26.733672 17643 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:26.741277 17643 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:26.743331 17643 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:26.746549 17800 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:26.749431 12034 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:26.751803 12034 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:26.752476 12033 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:26.753481 12033 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:26.757382 12033 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:26.757843 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:26.758389 17800 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:26.759387 17643 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:26.793331 17785 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:26.793962 17785 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:26.799996 11483 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:26.805213 11484 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:26.805635 11484 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:26.806391 17768 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:26.807229 17785 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:26.811152 12033 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:26.841126 12033 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:26.907160 12033 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:26.957247 12033 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:26.984941 11483 consensus_peers.cc:597] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.019821 17785 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.020346 17785 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:27.022315 17785 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.029265 11483 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.030686 12033 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.039744 17643 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.040349 17643 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:27.042479 17643 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.047796 12033 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:27.052165 12034 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:27.052743 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.053793 17800 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:27.054713 11484 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:27.060113 11483 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:27.060679 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.061712 17785 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:17:27.064131 11483 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.074676 11483 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.089020 17800 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.089592 17800 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:27.092142 17800 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.098436 12033 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:27.104558 12034 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:27.105103 12034 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:27.098832 11483 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.106019 17800 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:27.134117 12033 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.178647 11483 consensus_peers.cc:597] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.185985 11483 consensus_peers.cc:597] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.188395 11483 consensus_peers.cc:597] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.201403 12033 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.255926 17800 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.256505 17800 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:27.260880 17800 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.265048 11483 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:27.269438 11483 consensus_peers.cc:597] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:27.270994 12034 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:27.272367 12033 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:27.272876 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.273805 17800 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:27.314267 11483 consensus_peers.cc:597] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.344879 12033 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.367585 11483 consensus_peers.cc:597] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:27.376767 12033 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.389789 17800 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.390261 17800 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:27.392088 17800 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.396585 12034 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:27.399942 12033 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:27.400331 12033 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.400985 17800 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:27.418260 10257 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 11466)
W20260709 13:17:27.441087 12033 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:27.465802 11483 consensus_peers.cc:597] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:27.491354 17816 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.491910 17816 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:27.493911 17816 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.499711 11484 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:27.500969 11483 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:27.501339 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.502004 17816 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:17:27.566280 12033 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.616789 17816 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.618461 17816 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:27.627229 11484 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:27.625423 17816 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.628980 11483 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:27.629416 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.632259 17785 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:17:27.647238 11483 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:27.666090 11483 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:27.708974 17816 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:27.709503 17816 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:27.716382 11483 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:27.719647 17785 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
W20260709 13:17:27.721607 11484 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:27.721540 17785 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:27.723658 17785 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:27.720870 17816 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:27.730089 11483 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:27.723845 11484 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.736685 17785 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:27.743939 11484 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:27.749451 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:27.751052 17785 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:27.886379 11484 consensus_peers.cc:597] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:28.055822 11483 consensus_peers.cc:597] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.290295 12034 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.290931 12034 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.389109 12034 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.395315 12034 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.419185 10257 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 11466)
W20260709 13:17:28.423352 12034 consensus_peers.cc:597] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:28.443950 12033 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.516499 11484 consensus_peers.cc:597] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.516885 12034 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:28.537185 11484 consensus_peers.cc:597] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:28.566380 12034 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:28.571285 17643 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:28.571902 17643 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:28.573922 17643 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:28.584012 12034 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:28.584901 12033 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:28.585309 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:28.586993 17800 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:28.592638 12034 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:28.621628 17785 raft_consensus.cc:493] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:28.622309 17785 raft_consensus.cc:515] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:28.624377 17785 leader_election.cc:290] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:28.639343 11484 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:28.640239 11483 leader_election.cc:336] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:28.640650 11483 leader_election.cc:304] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:28.642637 17816 raft_consensus.cc:2749] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:17:28.674551 11484 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.684139 12034 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.684199 11484 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.694731 11484 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.747187 11484 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:28.753123 17643 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:28.753681 17643 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:28.760201 12033 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:28.761126 17643 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:28.766757 12034 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:28.767196 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:28.767917 17643 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:28.794957 12034 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.803879 12034 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.805423 11484 consensus_peers.cc:597] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.816952 11484 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.826607 12033 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:28.840422 12034 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.883966 12034 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.886258 11484 consensus_peers.cc:597] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.894398 12034 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.901352 11484 consensus_peers.cc:597] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.954990 11484 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.957804 12034 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:28.978016 12034 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:29.101658 11484 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:29.142140 11483 consensus_peers.cc:597] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:29.190657 17643 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.191169 17643 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:29.193113 17643 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:29.213068 12034 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:29.213960 12033 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:29.214358 12033 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:29.215859 17643 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:29.275612 17800 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.276175 17800 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:29.278278 17800 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:29.278733 17800 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.279161 17800 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:29.325615 17800 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:29.332162 12034 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:29.355618 12034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111) [suppressed 661 similar messages]
I20260709 13:17:29.363049 17768 raft_consensus.cc:493] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.372577 17819 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.378805 17768 raft_consensus.cc:515] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:29.378913 17819 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:29.380812 17768 leader_election.cc:290] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:29.381027 17819 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:29.388317 11483 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:37891, user_credentials={real_user=slave}} blocked reactor thread for 45753.4us
W20260709 13:17:29.419675 10257 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 11466)
W20260709 13:17:29.418015 12033 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:29.420398 12033 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:29.422806 17800 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:29.422115 12034 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:29.429306 11484 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:29.476286 11483 leader_election.cc:336] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:29.491958 11483 leader_election.cc:304] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:29.499930 17820 raft_consensus.cc:2749] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:17:29.498193 12034 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:29.540823 12034 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:29.544782 17785 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.549135 17785 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:29.550896 17785 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:29.556959 12033 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:29.563000 12033 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:29.571038 12033 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:29.575577 11484 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:29.575110 12033 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:29.580835 17800 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:29.581745 12033 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:29.587090 12033 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:29.586421 11483 consensus_peers.cc:597] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:29.596750 17800 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:29.598750 12033 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.629760 11483 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:29.641117 11483 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:29.649165 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:29.650665 12033 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.662175 12033 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:29.662724 17785 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:29.671494 12033 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.675974 12033 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:29.706511 11483 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.712538 11483 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.735138 12033 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.778517 11483 consensus_peers.cc:597] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:29.798226 12033 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.809890 11483 consensus_peers.cc:597] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:29.815456 17800 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.816041 17800 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:29.817967 17800 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:29.811828 12033 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.845173 12034 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:29.861784 11483 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.867817 12033 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:29.871276 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:29.878677 17800 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:29.888769 17785 raft_consensus.cc:493] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.889312 17785 raft_consensus.cc:515] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:29.891671 17785 leader_election.cc:290] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:29.896387 11484 leader_election.cc:336] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:29.919749 11483 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.920322 11483 consensus_peers.cc:597] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.933467 11483 consensus_peers.cc:597] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:29.934059 11483 leader_election.cc:336] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:29.934460 11483 leader_election.cc:304] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:29.935165 17785 raft_consensus.cc:2749] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:29.964697 11483 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.975840 12033 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:29.991575 11483 consensus_peers.cc:597] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:29.996418 17785 raft_consensus.cc:493] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:29.996887 17785 raft_consensus.cc:515] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:29.998731 17785 leader_election.cc:290] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:30.011477 11484 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:30.134249 12032 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:46667, user_credentials={real_user=slave}} blocked reactor thread for 47376.7us
W20260709 13:17:30.160175 11483 leader_election.cc:336] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:30.160709 11483 leader_election.cc:304] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:30.162022 17823 raft_consensus.cc:2749] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:17:30.184142 11483 consensus_peers.cc:597] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:30.188655 17823 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:30.189343 17823 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:30.197141 11484 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:30.197913 17823 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:30.207728 11483 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:30.216223 12033 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:30.222537 11483 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:30.223150 11483 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:30.223923 17823 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:30.236747 17800 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:30.237349 17800 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:30.239288 17800 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:30.246001 12034 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:30.254380 12033 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:30.259008 12033 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:30.260426 17800 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:30.401086 11483 consensus_peers.cc:597] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:30.420634 10257 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 11466)
W20260709 13:17:30.504240 11484 consensus_peers.cc:597] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:30.517022 12034 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:30.580740 17800 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:30.581369 17800 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:30.583442 17800 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:30.592980 12033 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:30.600414 12034 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:30.600936 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:30.602547 17822 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:30.643321 17785 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:30.643841 17785 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
W20260709 13:17:30.651484 11484 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:30.661701 11483 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:30.663846 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:30.664651 17826 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:30.662205 17785 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:30.919529 11484 consensus_peers.cc:597] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:30.950584 12034 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:30.970750 12034 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:30.973886 12034 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:31.029757 11484 consensus_peers.cc:597] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:31.042665 12034 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.046356 12033 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.057875 12034 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.096858 11484 consensus_peers.cc:597] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.118989 11484 consensus_peers.cc:597] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.173614 12034 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.187770 12034 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.196301 12034 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.212863 11484 consensus_peers.cc:597] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.238411 12034 consensus_peers.cc:597] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:31.307066 12034 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.316661 12033 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:31.320819 12034 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.410486 12034 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.418286 12034 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.418968 11483 consensus_peers.cc:597] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:31.493042 11484 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.493891 12034 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.521329 12034 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.523063 11484 consensus_peers.cc:597] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.572320 11484 consensus_peers.cc:597] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.597749 11484 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:31.630551 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
W20260709 13:17:31.706589 11484 consensus_peers.cc:597] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.746737 12033 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:31.797289 11484 consensus_peers.cc:597] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:31.798291 17826 raft_consensus.cc:493] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:31.798923 17826 raft_consensus.cc:515] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:31.801137 17826 leader_election.cc:290] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:31.801978 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
W20260709 13:17:31.818130 12032 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:31.819684 12032 consensus_peers.cc:597] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:31.825655 11484 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:31.827759 12033 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:31.836068 12033 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:31.837399 11484 consensus_peers.cc:597] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:31.840586 11483 leader_election.cc:336] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:31.841104 11483 leader_election.cc:304] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:31.841863 17826 raft_consensus.cc:2749] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:31.875787 11483 consensus_peers.cc:597] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:31.905951 17826 raft_consensus.cc:493] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:31.906471 17826 raft_consensus.cc:515] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:31.908664 17826 leader_election.cc:290] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:31.915661 11483 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:31.919322 11484 leader_election.cc:336] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:31.919915 11484 leader_election.cc:304] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
W20260709 13:17:31.920710 12032 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.921188 17826 raft_consensus.cc:2749] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:31.922055 12033 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:31.936990 12033 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:31.942452 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:17:31.945147 12032 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.949038 10257 tablet_replica.cc:333] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.949986 10257 raft_consensus.cc:2243] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:31.951090 10257 raft_consensus.cc:2272] T e6e7edf5f82e49158a4023ea61d78284 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.953822 10257 tablet_replica.cc:333] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.959671 10257 raft_consensus.cc:2243] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:31.960350 10257 raft_consensus.cc:2272] T 1a992bbbf8dd4ccbb70ff8d65e31edae P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:31.962651 12032 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.963109 10257 tablet_replica.cc:333] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.963980 10257 raft_consensus.cc:2243] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:31.964624 10257 raft_consensus.cc:2272] T a38969de5c824718aa81b9622624bc51 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.967653 10257 tablet_replica.cc:333] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.968514 10257 raft_consensus.cc:2243] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:31.969604 10257 raft_consensus.cc:2272] T 7dab116931a64024ac61a35a6d29a836 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.972350 10257 tablet_replica.cc:333] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.973192 10257 raft_consensus.cc:2243] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:31.974233 10257 raft_consensus.cc:2272] T cf199b70fbb94d728a09194d18ca3b1e P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.976934 10257 tablet_replica.cc:333] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.977892 10257 raft_consensus.cc:2243] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:17:31.979090 10257 raft_consensus.cc:2272] T bab171deb5e946c8b074a4c091542a78 P 34a42aa6aa0a4778a532c6433d70b981 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.981802 10257 tablet_replica.cc:333] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.982642 10257 raft_consensus.cc:2243] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:31.983436 12033 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:31.983434 10257 raft_consensus.cc:2272] T c753736d0d7f4578b8935b855a6b36e3 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.986510 10257 tablet_replica.cc:333] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.987334 10257 raft_consensus.cc:2243] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:31.988055 10257 raft_consensus.cc:2272] T 7074ddeb0b7b4c1395b798352ff7db0a P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.990550 10257 tablet_replica.cc:333] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.991374 10257 raft_consensus.cc:2243] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:31.992290 10257 raft_consensus.cc:2272] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:31.995123 10257 tablet_replica.cc:333] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:31.995908 10257 raft_consensus.cc:2243] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:31.996465 10257 raft_consensus.cc:2272] T 887116b7f3e447028e317f03386bf8d0 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:31.997618 12032 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.999219 10257 tablet_replica.cc:333] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.000173 10257 raft_consensus.cc:2243] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:32.000481 11483 consensus_peers.cc:597] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:32.000767 10257 raft_consensus.cc:2272] T 683aea8d9bd64288b5932ad265b60dd4 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.003544 10257 tablet_replica.cc:333] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.004345 10257 raft_consensus.cc:2243] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.005403 10257 raft_consensus.cc:2272] T 6f3368fb0be64c3b92da009af48a1854 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:32.015893 12032 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.028035 12033 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:32.051137 12032 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.136158 12033 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:32.119310 10257 tablet_replica.cc:333] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.152408 10257 raft_consensus.cc:2243] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.153124 10257 raft_consensus.cc:2272] T 7a5787304dfe4697baeaeb367ee7dda3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.173779 10257 tablet_replica.cc:333] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.174765 10257 raft_consensus.cc:2243] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.175886 10257 raft_consensus.cc:2272] T 116f1226682347599c239ad3bee95b41 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.178622 10257 tablet_replica.cc:333] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.179165 10257 raft_consensus.cc:2243] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.180133 10257 raft_consensus.cc:2272] T 572d79fba792428db7fcc1994fff81f7 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.184173 10257 tablet_replica.cc:333] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.184948 10257 raft_consensus.cc:2243] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.185587 10257 raft_consensus.cc:2272] T 1ea18eac305644ed8d44618c2c0a8d0d P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.187510 10257 tablet_replica.cc:333] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.188028 10257 raft_consensus.cc:2243] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.188587 10257 raft_consensus.cc:2272] T ab2785abe6bd4898ac2dbb25f53e4c8b P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.191028 10257 tablet_replica.cc:333] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.191597 10257 raft_consensus.cc:2243] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.192054 10257 raft_consensus.cc:2272] T e5b7e70b18df44b6b0e248fbc3664d96 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.194079 10257 tablet_replica.cc:333] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.194589 10257 raft_consensus.cc:2243] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.195393 10257 raft_consensus.cc:2272] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.197731 10257 tablet_replica.cc:333] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.198447 10257 raft_consensus.cc:2243] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.199012 17828 raft_consensus.cc:493] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:32.199676 10257 raft_consensus.cc:2272] T 81ba42e5875044cb8ebec6e017d5585b P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.199595 17828 raft_consensus.cc:515] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:32.201719 10257 tablet_replica.cc:333] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.202533 10257 raft_consensus.cc:2243] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.203356 10257 raft_consensus.cc:2272] T 98fc3c2677604f3bb787b67c5e5b6012 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.205956 10257 tablet_replica.cc:333] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
W20260709 13:17:32.205955 12033 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:32.201807 17828 leader_election.cc:290] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:32.213675 10257 raft_consensus.cc:2243] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.214489 10257 raft_consensus.cc:2272] T 47922bb832834e72bc79e1e8159e6c1f P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.217428 10257 tablet_replica.cc:333] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.218475 10257 raft_consensus.cc:2243] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.223515 10257 raft_consensus.cc:2272] T 114b7c3bbf65457b98b17b61f3c64839 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.221998 17829 raft_consensus.cc:493] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:32.225373 17829 raft_consensus.cc:515] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } }
I20260709 13:17:32.232478 17829 leader_election.cc:290] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:32.255801 12032 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.314092 11484 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:32.327132 10257 tablet_replica.cc:333] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.328258 10257 raft_consensus.cc:2243] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:17:32.381515 12033 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:32.406350 10257 raft_consensus.cc:2272] T bcf62f83a1754dcc80291daa521a9c06 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.434566 10257 tablet_replica.cc:333] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.439193 10257 raft_consensus.cc:2243] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:32.445977 11483 consensus_peers.cc:597] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:32.453070 11483 consensus_peers.cc:597] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:32.453845 11483 leader_election.cc:336] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:32.454443 11483 leader_election.cc:304] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34a42aa6aa0a4778a532c6433d70b981; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:32.461066 17831 raft_consensus.cc:2749] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:32.466643 10257 raft_consensus.cc:2272] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.484352 10257 tablet_replica.cc:333] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.486624 10257 raft_consensus.cc:2243] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.491025 10257 raft_consensus.cc:2272] T 1073b5e444bf4b34ad3dca3ea96efc76 P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:32.491645 11483 consensus_peers.cc:597] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:32.496299 10257 tablet_replica.cc:333] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.497838 10257 raft_consensus.cc:2243] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.498915 10257 raft_consensus.cc:2272] T df8439ad70bf422daa65806caf52f7f4 P 34a42aa6aa0a4778a532c6433d70b981 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.507455 10257 tablet_replica.cc:333] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.508774 10257 raft_consensus.cc:2243] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.509738 10257 raft_consensus.cc:2272] T dbbdba17cce24fe7923cf755be2d3e0a P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.513345 10257 tablet_replica.cc:333] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.514523 10257 raft_consensus.cc:2243] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.517171 10257 raft_consensus.cc:2272] T 8dd69c97ea4640a0a71117011ab4e5d4 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.520449 10257 tablet_replica.cc:333] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.521392 10257 raft_consensus.cc:2243] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.522938 10257 raft_consensus.cc:2272] T e4aa2e377a0f45378e0df4a788c07ddf P 34a42aa6aa0a4778a532c6433d70b981 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.526503 10257 tablet_replica.cc:333] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.527472 10257 raft_consensus.cc:2243] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Raft consensus shutting down.
W20260709 13:17:32.528040 12034 leader_election.cc:336] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:32.528613 12034 leader_election.cc:304] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:32.529999 10257 raft_consensus.cc:2272] T b01b3301ee794d99bcb4feb672ac699f P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.534734 10257 tablet_replica.cc:333] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.535555 17835 raft_consensus.cc:2749] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:32.535758 10257 raft_consensus.cc:2243] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.536851 10257 raft_consensus.cc:2272] T 988262f8a4a846be8b350d9d99c4a730 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.540177 10257 tablet_replica.cc:333] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.541352 10257 raft_consensus.cc:2243] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.542424 10257 raft_consensus.cc:2272] T 64ec4a2bcc7f4fe1851f6d77e7515fed P 34a42aa6aa0a4778a532c6433d70b981 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:32.553592 12034 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:32.560487 12033 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:32.562099 10257 tablet_replica.cc:333] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.562875 10257 raft_consensus.cc:2243] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.563907 10257 raft_consensus.cc:2272] T 7d370e878a484e1aaaa0df1466da6f24 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.566715 10257 tablet_replica.cc:333] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.567430 10257 raft_consensus.cc:2243] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:17:32.567618 12032 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.568472 12032 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.569063 10257 raft_consensus.cc:2272] T 78b9049e00f94cc1912970768037cf24 P 34a42aa6aa0a4778a532c6433d70b981 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:32.569170 12032 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.569932 12032 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.571295 12032 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.572268 12032 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.572500 10257 tablet_replica.cc:333] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
W20260709 13:17:32.572860 12032 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.573441 10257 raft_consensus.cc:2243] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.575492 10257 raft_consensus.cc:2272] T 3cd0c27e51bd41b5959c50a56353166e P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.585955 10257 tablet_replica.cc:333] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.586977 10257 raft_consensus.cc:2243] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.588775 10257 raft_consensus.cc:2272] T 43fc774e6f3a42cf890bd8e9fb8f7635 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.592075 10257 tablet_replica.cc:333] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.592876 10257 raft_consensus.cc:2243] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.593822 10257 raft_consensus.cc:2272] T 2eaec0f849964172b0dcfba0af0e7fa3 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:32.599942 12033 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:32.602139 10257 tablet_replica.cc:333] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.603108 10257 raft_consensus.cc:2243] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.604710 10257 raft_consensus.cc:2272] T 063871012da74857a239cdf01c76e5ab P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.608042 10257 tablet_replica.cc:333] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.608819 10257 raft_consensus.cc:2243] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.610395 10257 raft_consensus.cc:2272] T 593d6a54177d461680000f43998f8775 P 34a42aa6aa0a4778a532c6433d70b981 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.613787 10257 tablet_replica.cc:333] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.614498 10257 raft_consensus.cc:2243] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.616201 10257 raft_consensus.cc:2272] T 44cbd38458af4b03b0d3f362bd6a2f85 P 34a42aa6aa0a4778a532c6433d70b981 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.619678 10257 tablet_replica.cc:333] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.620464 10257 raft_consensus.cc:2243] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.622152 10257 raft_consensus.cc:2272] T 6b402b68ac7b4b9292a6ab6fb6fd3ea4 P 34a42aa6aa0a4778a532c6433d70b981 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.625260 10257 tablet_replica.cc:333] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.625936 10257 raft_consensus.cc:2243] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:32.626672 10257 raft_consensus.cc:2272] T ae095c448d6e4a3ab9eaecd9695887a7 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.629750 10257 tablet_replica.cc:333] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.630398 10257 raft_consensus.cc:2243] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.631932 10257 raft_consensus.cc:2272] T 741facf7b8264a10bc5c3431415fc34a P 34a42aa6aa0a4778a532c6433d70b981 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:32.634877 10257 tablet_replica.cc:333] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981: stopping tablet replica
I20260709 13:17:32.635516 10257 raft_consensus.cc:2243] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:32.637096 10257 raft_consensus.cc:2272] T a7effda9a8a1409cbe25d55ed7cf2964 P 34a42aa6aa0a4778a532c6433d70b981 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:32.647301 12033 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:32.672080 17835 raft_consensus.cc:493] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:32.672870 17835 raft_consensus.cc:515] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:32.675657 17835 leader_election.cc:290] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:32.680095 12033 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:32.684240 12034 leader_election.cc:336] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:32.684741 12034 leader_election.cc:304] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:32.685621 17834 raft_consensus.cc:2749] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:33.004637 17800 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:33.005270 17800 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.007463 17800 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:33.012497 12033 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.014109 12034 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:33.014541 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.015241 17800 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:33.127907 17834 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.128895 17834 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.131249 17834 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.139998 12032 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
W20260709 13:17:33.140949 12034 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:33.141396 12034 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.142783 17834 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:33.143599 17800 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.144199 17800 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.146234 17800 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.150251 12034 leader_election.cc:336] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:33.151300 12032 leader_election.cc:336] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.151705 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.152431 17834 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:33.175627 12034 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.211386 11540 debug-util.cc:398] Leaking SignalData structure 0x7b08002e7700 after lost signal to thread 10260
W20260709 13:17:33.212886 11540 debug-util.cc:398] Leaking SignalData structure 0x7b08000ca2a0 after lost signal to thread 11302
W20260709 13:17:33.214680 11540 debug-util.cc:398] Leaking SignalData structure 0x7b080037e1a0 after lost signal to thread 12093
I20260709 13:17:33.227384 17800 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.228060 17800 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.230042 17800 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.234304 12034 leader_election.cc:336] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:33.239493 12032 leader_election.cc:336] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.240006 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.240737 17800 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:33.264663 17800 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.265281 17800 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.266983 17800 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.271752 12034 leader_election.cc:336] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:33.276412 12032 leader_election.cc:336] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.276865 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.277568 17800 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:33.279552 17800 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.279991 17800 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.282608 17800 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.286667 12033 leader_election.cc:336] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.292243 12032 leader_election.cc:336] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.292784 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:33.293586 17800 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:33.350880 17800 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.351610 17800 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:33.357635 12034 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:33.358777 17800 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:33.359452 17834 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.360603 17834 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.362697 17834 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.368204 12033 leader_election.cc:336] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.374076 12032 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.374572 12032 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.375355 17800 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:33.376399 12032 leader_election.cc:336] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.376821 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:33.377566 17800 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:33.439889 17800 raft_consensus.cc:493] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:33.440431 17800 raft_consensus.cc:515] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.442354 17800 leader_election.cc:290] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:33.451256 17834 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.451762 17834 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.453709 17834 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
W20260709 13:17:33.455189 12034 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:33.458974 12033 leader_election.cc:336] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:33.460011 12033 leader_election.cc:304] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.463339 17800 raft_consensus.cc:2749] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:33.470201 12033 leader_election.cc:336] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:33.479043 17800 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:33.479918 17800 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.482015 17800 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:33.493912 12033 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.501722 12032 leader_election.cc:336] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.503655 12032 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
W20260709 13:17:33.509025 12034 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:33.511513 17800 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:33.515106 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.520973 17800 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:33.542165 12034 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:33.542742 12034 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.543179 12034 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:33.570029 17834 raft_consensus.cc:493] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:33.570613 17834 raft_consensus.cc:515] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.572780 17834 leader_election.cc:290] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:33.583796 12033 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.586738 12034 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.587313 12034 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.590247 12034 leader_election.cc:336] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:33.590677 12034 leader_election.cc:304] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.591434 17800 raft_consensus.cc:2749] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:33.621094 12033 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.646852 12034 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.688738 12034 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.768018 10257 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 11540)
I20260709 13:17:33.829507 17800 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.830222 17800 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.833244 17800 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.838841 12034 leader_election.cc:336] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:33.840195 17800 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.840759 17800 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.844825 17800 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.852938 12033 leader_election.cc:336] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.855847 12032 leader_election.cc:336] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.856365 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:33.857532 17800 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:33.860677 12032 leader_election.cc:336] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.861235 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:33.863346 17837 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.863897 17837 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.865809 17837 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.871997 12034 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:33.879562 17800 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:17:33.881870 12033 leader_election.cc:336] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.885511 12034 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:33.910099 12032 leader_election.cc:336] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.910665 12032 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:33.911439 17800 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:33.924896 17800 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:33.925448 17800 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:33.927203 17800 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:33.932791 12033 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:33.936643 12032 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:33.937120 12032 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:33.937814 17800 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:34.115193 17800 raft_consensus.cc:493] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:34.115897 17800 raft_consensus.cc:515] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:34.117993 17800 leader_election.cc:290] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:34.121752 17834 raft_consensus.cc:493] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:34.122339 17834 raft_consensus.cc:515] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:34.124411 17834 leader_election.cc:290] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:34.257441 17834 raft_consensus.cc:493] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:34.258090 17834 raft_consensus.cc:515] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:34.260161 17834 leader_election.cc:290] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
I20260709 13:17:34.269095 17834 raft_consensus.cc:493] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:34.269686 17834 raft_consensus.cc:515] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:34.271672 17834 leader_election.cc:290] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:34.310989 12034 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:34.350183 12033 leader_election.cc:336] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:34.350899 12033 leader_election.cc:336] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:34.375562 12034 consensus_peers.cc:597] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:34.376159 12034 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:34.376581 12034 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:34.429920 12033 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111) [suppressed 584 similar messages]
W20260709 13:17:34.431360 12033 leader_election.cc:336] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:34.431879 12033 leader_election.cc:304] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:34.432680 17834 raft_consensus.cc:2749] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:34.476814 12033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:42857, user_credentials={real_user=slave}} blocked reactor thread for 45468us
W20260709 13:17:34.477710 12033 leader_election.cc:336] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:34.577322 12034 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:34.577838 12034 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:34.577310 12033 consensus_peers.cc:597] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:34.579051 12033 consensus_peers.cc:597] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:34.594551 12034 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:34.601032 12033 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:34.607724 12033 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:34.608333 12033 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:34.614346 12032 leader_election.cc:336] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:34.614910 12032 leader_election.cc:304] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:34.615702 17843 raft_consensus.cc:2749] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:34.616176 12032 leader_election.cc:336] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:34.616644 12032 leader_election.cc:304] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:34.617303 17843 raft_consensus.cc:2749] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:17:34.621259 12032 consensus_peers.cc:597] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:34.621910 12032 leader_election.cc:336] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:34.622279 12032 leader_election.cc:304] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:34.622977 17843 raft_consensus.cc:2749] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:17:34.692838 12032 consensus_peers.cc:597] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:34.702564 12032 consensus_peers.cc:597] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:34.721386 12032 consensus_peers.cc:597] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:34.748359 17842 raft_consensus.cc:493] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:34.748845 17842 raft_consensus.cc:515] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:34.750783 17842 leader_election.cc:290] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:34.755231 12032 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
W20260709 13:17:34.755880 12034 leader_election.cc:336] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:34.756187 12034 leader_election.cc:304] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:34.756757 17842 raft_consensus.cc:2749] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:34.768515 10257 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 11540)
W20260709 13:17:34.791595 12033 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:34.833891 12032 consensus_peers.cc:597] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:34.958019 17842 raft_consensus.cc:493] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:34.958546 17842 raft_consensus.cc:515] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:34.960508 17842 leader_election.cc:290] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:34.965823 12034 leader_election.cc:336] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:34.972841 12032 leader_election.cc:336] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:34.973356 12032 leader_election.cc:304] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:34.974069 17800 raft_consensus.cc:2749] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:34.975023 12033 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:34.985913 12033 consensus_peers.cc:597] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:34.987172 12032 consensus_peers.cc:597] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:34.987228 12033 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:34.997973 12032 consensus_peers.cc:597] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.011550 12032 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.021624 12034 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:35.083554 12033 consensus_peers.cc:597] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:35.086172 12032 consensus_peers.cc:597] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.095986 12033 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:35.111450 12032 consensus_peers.cc:597] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.148816 12032 consensus_peers.cc:597] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.151186 12032 consensus_peers.cc:597] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.199753 12032 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:35.231657 17842 raft_consensus.cc:493] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.232219 17842 raft_consensus.cc:515] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.234093 17842 leader_election.cc:290] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.238518 12034 leader_election.cc:336] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:35.239117 12032 leader_election.cc:336] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:35.239554 12032 leader_election.cc:304] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:35.240226 17800 raft_consensus.cc:2749] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:35.259492 12033 consensus_peers.cc:597] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:17:35.270362 17800 raft_consensus.cc:493] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.270881 17800 raft_consensus.cc:515] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.272886 17800 leader_election.cc:290] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.276485 12033 leader_election.cc:336] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:35.278077 12032 leader_election.cc:336] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:35.278404 12032 leader_election.cc:304] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:35.278918 17800 raft_consensus.cc:2749] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:17:35.332288 12033 consensus_peers.cc:597] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:35.342415 17800 raft_consensus.cc:493] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.342970 17800 raft_consensus.cc:515] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.344738 17800 leader_election.cc:290] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667), 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857)
W20260709 13:17:35.350703 12032 leader_election.cc:336] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
W20260709 13:17:35.355134 12033 leader_election.cc:336] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:35.355543 12033 leader_election.cc:304] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:35.356091 17800 raft_consensus.cc:2749] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:35.369302 17800 raft_consensus.cc:493] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.369745 17800 raft_consensus.cc:515] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.371693 17800 leader_election.cc:290] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.376441 12034 leader_election.cc:336] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
W20260709 13:17:35.378115 12032 leader_election.cc:336] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:35.378511 12032 leader_election.cc:304] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:35.379547 17800 raft_consensus.cc:2749] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:17:35.388180 17800 raft_consensus.cc:493] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.388687 17800 raft_consensus.cc:515] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.390569 17800 leader_election.cc:290] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.395579 12033 leader_election.cc:336] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:35.396262 12032 consensus_peers.cc:597] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.396634 12032 leader_election.cc:336] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:35.396915 12032 leader_election.cc:304] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:35.397514 17800 raft_consensus.cc:2749] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:35.462052 17800 raft_consensus.cc:493] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.463276 17800 raft_consensus.cc:515] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.465287 17800 leader_election.cc:290] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.471931 12033 leader_election.cc:336] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:35.472968 12032 leader_election.cc:336] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:35.473285 12032 leader_election.cc:304] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:35.473939 17842 raft_consensus.cc:2749] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:35.478580 17842 raft_consensus.cc:493] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.478960 17842 raft_consensus.cc:515] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.480777 17842 leader_election.cc:290] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.485255 12032 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
W20260709 13:17:35.485672 12033 leader_election.cc:336] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:35.486006 12033 leader_election.cc:304] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:35.486526 17842 raft_consensus.cc:2749] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:17:35.492549 17842 raft_consensus.cc:493] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.492957 17842 raft_consensus.cc:515] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:35.493443 12090 debug-util.cc:398] Leaking SignalData structure 0x7b08003db4a0 after lost signal to thread 10260
W20260709 13:17:35.494508 12090 debug-util.cc:398] Leaking SignalData structure 0x7b08003ae680 after lost signal to thread 11302
I20260709 13:17:35.494941 17842 leader_election.cc:290] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.495890 12090 debug-util.cc:398] Leaking SignalData structure 0x7b0800221c00 after lost signal to thread 12093
W20260709 13:17:35.501196 12032 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
W20260709 13:17:35.506767 12034 leader_election.cc:336] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:35.507254 12034 leader_election.cc:304] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:35.509904 17842 raft_consensus.cc:2749] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:17:35.565599 12032 consensus_peers.cc:597] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:35.623175 12032 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Couldn't send request to peer 34a42aa6aa0a4778a532c6433d70b981. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:35.696764 17800 raft_consensus.cc:493] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.697278 17800 raft_consensus.cc:515] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:35.699184 17800 leader_election.cc:290] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
W20260709 13:17:35.711156 12034 leader_election.cc:336] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:35.734639 17842 raft_consensus.cc:493] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 34a42aa6aa0a4778a532c6433d70b981)
I20260709 13:17:35.735349 17842 raft_consensus.cc:515] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "34a42aa6aa0a4778a532c6433d70b981" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 46667 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:35.743368 12033 leader_election.cc:336] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:35.745167 12032 leader_election.cc:336] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:35.750145 12032 leader_election.cc:304] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 34a42aa6aa0a4778a532c6433d70b981, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:35.749482 17842 leader_election.cc:290] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667)
I20260709 13:17:35.753280 17800 raft_consensus.cc:2749] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:35.775840 12032 leader_election.cc:336] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 34a42aa6aa0a4778a532c6433d70b981 (127.10.4.67:46667): Network error: Client connection negotiation failed: client connection to 127.10.4.67:46667: connect: Connection refused (error 111)
I20260709 13:17:35.776561 12032 leader_election.cc:304] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 34a42aa6aa0a4778a532c6433d70b981
I20260709 13:17:35.778379 17842 raft_consensus.cc:2749] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:17:36.018005 12034 consensus_peers.cc:597] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:36.081540 12034 consensus_peers.cc:597] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 -> Peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Couldn't send request to peer 6e87b213a4554d94825aceec352fc094. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:36.112149 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
W20260709 13:17:36.133019 12033 consensus_peers.cc:597] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 -> Peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Couldn't send request to peer 01ae50c877d04ba687e4e06259ba5581. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:36.204075 17800 raft_consensus.cc:493] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:36.204576 17800 raft_consensus.cc:515] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:36.207449 17800 leader_election.cc:290] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:36.214980 12033 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:36.232398 12034 leader_election.cc:336] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:36.232903 12034 leader_election.cc:304] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:36.234717 17842 raft_consensus.cc:2749] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:17:36.288439 17842 raft_consensus.cc:493] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:36.289175 17842 raft_consensus.cc:515] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
I20260709 13:17:36.292742 17842 leader_election.cc:290] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:36.299436 12033 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
W20260709 13:17:36.310648 12034 leader_election.cc:336] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:36.311249 12034 leader_election.cc:304] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:36.312074 17842 raft_consensus.cc:2749] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:36.313570 17842 raft_consensus.cc:493] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6e87b213a4554d94825aceec352fc094)
I20260709 13:17:36.314179 17842 raft_consensus.cc:515] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01ae50c877d04ba687e4e06259ba5581" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 42857 } } peers { permanent_uuid: "6e87b213a4554d94825aceec352fc094" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 44257 } } peers { permanent_uuid: "b1906ab37ba541f493936685a808c8a9" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 37891 } attrs { promote: false } }
W20260709 13:17:36.321789 12033 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857): Network error: Client connection negotiation failed: client connection to 127.10.4.65:42857: connect: Connection refused (error 111)
I20260709 13:17:36.323119 17842 leader_election.cc:290] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01ae50c877d04ba687e4e06259ba5581 (127.10.4.65:42857), 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257)
W20260709 13:17:36.328960 12034 leader_election.cc:336] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6e87b213a4554d94825aceec352fc094 (127.10.4.66:44257): Network error: Client connection negotiation failed: client connection to 127.10.4.66:44257: connect: Connection refused (error 111)
I20260709 13:17:36.329504 12034 leader_election.cc:304] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1906ab37ba541f493936685a808c8a9; no voters: 01ae50c877d04ba687e4e06259ba5581, 6e87b213a4554d94825aceec352fc094
I20260709 13:17:36.329953 10257 tablet_server.cc:179] TabletServer@127.10.4.68:0 shutting down...
I20260709 13:17:36.330304 17842 raft_consensus.cc:2749] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:17:36.383500 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:36.384856 10257 tablet_replica.cc:333] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.385900 10257 raft_consensus.cc:2243] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.386615 10257 raft_consensus.cc:2272] T 81ba42e5875044cb8ebec6e017d5585b P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.389873 10257 tablet_replica.cc:333] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.390745 10257 raft_consensus.cc:2243] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.392380 10257 raft_consensus.cc:2272] T 1073b5e444bf4b34ad3dca3ea96efc76 P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.395612 10257 tablet_replica.cc:333] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.396485 10257 raft_consensus.cc:2243] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.397972 10257 raft_consensus.cc:2272] T 7d370e878a484e1aaaa0df1466da6f24 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.400857 10257 tablet_replica.cc:333] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.401618 10257 raft_consensus.cc:2243] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.403110 10257 raft_consensus.cc:2272] T 7a5787304dfe4697baeaeb367ee7dda3 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.406615 10257 tablet_replica.cc:333] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.407529 10257 raft_consensus.cc:2243] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.408335 10257 raft_consensus.cc:2272] T 8a142ac56caa42a08e0a838dc87ac3b3 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.411427 10257 tablet_replica.cc:333] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.412238 10257 raft_consensus.cc:2243] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.413635 10257 raft_consensus.cc:2272] T 1a992bbbf8dd4ccbb70ff8d65e31edae P b1906ab37ba541f493936685a808c8a9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.416893 10257 tablet_replica.cc:333] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.417943 10257 raft_consensus.cc:2243] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.418617 10257 raft_consensus.cc:2272] T b8ec7d9e61ab4ff5bd59a37021d68dba P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.421633 10257 tablet_replica.cc:333] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.422747 10257 raft_consensus.cc:2243] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.423483 10257 raft_consensus.cc:2272] T b01b3301ee794d99bcb4feb672ac699f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.426436 10257 tablet_replica.cc:333] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.427337 10257 raft_consensus.cc:2243] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.428651 10257 raft_consensus.cc:2272] T 988262f8a4a846be8b350d9d99c4a730 P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.431610 10257 tablet_replica.cc:333] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.432493 10257 raft_consensus.cc:2243] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.433754 10257 raft_consensus.cc:2272] T dbbdba17cce24fe7923cf755be2d3e0a P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.436640 10257 tablet_replica.cc:333] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.437412 10257 raft_consensus.cc:2243] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.438103 10257 raft_consensus.cc:2272] T 38ecdb2d3e8745889c0343749502c9f5 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.440872 10257 tablet_replica.cc:333] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.441998 10257 raft_consensus.cc:2243] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.442761 10257 raft_consensus.cc:2272] T 572d79fba792428db7fcc1994fff81f7 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.445891 10257 tablet_replica.cc:333] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.446770 10257 raft_consensus.cc:2243] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.447499 10257 raft_consensus.cc:2272] T 1ea18eac305644ed8d44618c2c0a8d0d P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.450210 10257 tablet_replica.cc:333] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.451236 10257 raft_consensus.cc:2243] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.452530 10257 raft_consensus.cc:2272] T bce024e6afd54cf0992c0af40df6a4eb P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.455451 10257 tablet_replica.cc:333] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.456285 10257 raft_consensus.cc:2243] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.456952 10257 raft_consensus.cc:2272] T 81760accce3a440a82323bd3ff53c551 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.460008 10257 tablet_replica.cc:333] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.460793 10257 raft_consensus.cc:2243] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.463389 10257 raft_consensus.cc:2272] T e5b7e70b18df44b6b0e248fbc3664d96 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.466482 10257 tablet_replica.cc:333] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.467338 10257 raft_consensus.cc:2243] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.468537 10257 raft_consensus.cc:2272] T ad3ab48ae4bf4452a19c99c71fd014f1 P b1906ab37ba541f493936685a808c8a9 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.471175 10257 tablet_replica.cc:333] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.472011 10257 raft_consensus.cc:2243] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.473141 10257 raft_consensus.cc:2272] T 6afc1298d03c4fa6a7ce8e41231e8cf4 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.475744 10257 tablet_replica.cc:333] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.476542 10257 raft_consensus.cc:2243] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.477249 10257 raft_consensus.cc:2272] T cf199b70fbb94d728a09194d18ca3b1e P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.480120 10257 tablet_replica.cc:333] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.480924 10257 raft_consensus.cc:2243] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.482136 10257 raft_consensus.cc:2272] T 9c1dd88ce9a04ff59ee55a115686f148 P b1906ab37ba541f493936685a808c8a9 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.485404 10257 tablet_replica.cc:333] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.486250 10257 raft_consensus.cc:2243] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.487203 10257 raft_consensus.cc:2272] T 47922bb832834e72bc79e1e8159e6c1f P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.490286 10257 tablet_replica.cc:333] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.491163 10257 raft_consensus.cc:2243] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.492089 10257 raft_consensus.cc:2272] T 99667fcf16014dc1b9b174f0a37d16af P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.495075 10257 tablet_replica.cc:333] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.495927 10257 raft_consensus.cc:2243] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.496631 10257 raft_consensus.cc:2272] T 857937fa61dc400a81345944f0a14511 P b1906ab37ba541f493936685a808c8a9 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.499635 10257 tablet_replica.cc:333] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.500509 10257 raft_consensus.cc:2243] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.501789 10257 raft_consensus.cc:2272] T 887116b7f3e447028e317f03386bf8d0 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.504653 10257 tablet_replica.cc:333] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.505434 10257 raft_consensus.cc:2243] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.506309 10257 raft_consensus.cc:2272] T 7dab116931a64024ac61a35a6d29a836 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.509093 10257 tablet_replica.cc:333] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.509989 10257 raft_consensus.cc:2243] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.510699 10257 raft_consensus.cc:2272] T e6e7edf5f82e49158a4023ea61d78284 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.513633 10257 tablet_replica.cc:333] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.514459 10257 raft_consensus.cc:2243] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.515189 10257 raft_consensus.cc:2272] T 116f1226682347599c239ad3bee95b41 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.518036 10257 tablet_replica.cc:333] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.518792 10257 raft_consensus.cc:2243] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.519505 10257 raft_consensus.cc:2272] T 78b9049e00f94cc1912970768037cf24 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.522204 10257 tablet_replica.cc:333] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.522940 10257 raft_consensus.cc:2243] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.523962 10257 raft_consensus.cc:2272] T 64ec4a2bcc7f4fe1851f6d77e7515fed P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.526512 10257 tablet_replica.cc:333] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.527261 10257 raft_consensus.cc:2243] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.528628 10257 raft_consensus.cc:2272] T 98fc3c2677604f3bb787b67c5e5b6012 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.531425 10257 tablet_replica.cc:333] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.532169 10257 raft_consensus.cc:2243] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.533334 10257 raft_consensus.cc:2272] T 4ad5a541019548a3b543619a92b420b1 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.535912 10257 tablet_replica.cc:333] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.536633 10257 raft_consensus.cc:2243] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.537758 10257 raft_consensus.cc:2272] T c753736d0d7f4578b8935b855a6b36e3 P b1906ab37ba541f493936685a808c8a9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.542873 10257 tablet_replica.cc:333] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.543622 10257 raft_consensus.cc:2243] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.545190 10257 raft_consensus.cc:2272] T 54c0e900e01b4e5aa4cf2fa0acbf6e86 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.548257 10257 tablet_replica.cc:333] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.548954 10257 raft_consensus.cc:2243] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.550158 10257 raft_consensus.cc:2272] T 7074ddeb0b7b4c1395b798352ff7db0a P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.553196 10257 tablet_replica.cc:333] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.553965 10257 raft_consensus.cc:2243] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.556028 10257 raft_consensus.cc:2272] T 11c8fff0a9ad4f4c9b41c65f74776fd6 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.558790 10257 tablet_replica.cc:333] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.559532 10257 raft_consensus.cc:2243] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.560261 10257 raft_consensus.cc:2272] T bcf62f83a1754dcc80291daa521a9c06 P b1906ab37ba541f493936685a808c8a9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.562877 10257 tablet_replica.cc:333] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.563670 10257 raft_consensus.cc:2243] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.564304 10257 raft_consensus.cc:2272] T 435f2e223e784d188897ac4adbc34211 P b1906ab37ba541f493936685a808c8a9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.566926 10257 tablet_replica.cc:333] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.567653 10257 raft_consensus.cc:2243] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.568234 10257 raft_consensus.cc:2272] T 063871012da74857a239cdf01c76e5ab P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.570852 10257 tablet_replica.cc:333] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.571542 10257 raft_consensus.cc:2243] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.572204 10257 raft_consensus.cc:2272] T 6002d9cfb1c74b70b8ec301e31e4bbe9 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.574966 10257 tablet_replica.cc:333] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.575736 10257 raft_consensus.cc:2243] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.576251 10257 raft_consensus.cc:2272] T e4aa2e377a0f45378e0df4a788c07ddf P b1906ab37ba541f493936685a808c8a9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.578825 10257 tablet_replica.cc:333] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.579521 10257 raft_consensus.cc:2243] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.580881 10257 raft_consensus.cc:2272] T 8b949bd9634b40af97ef4fb06ee21e9f P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.583712 10257 tablet_replica.cc:333] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.584388 10257 raft_consensus.cc:2243] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.585477 10257 raft_consensus.cc:2272] T ae095c448d6e4a3ab9eaecd9695887a7 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.588234 10257 tablet_replica.cc:333] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.588884 10257 raft_consensus.cc:2243] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.589560 10257 raft_consensus.cc:2272] T a7effda9a8a1409cbe25d55ed7cf2964 P b1906ab37ba541f493936685a808c8a9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.592414 10257 tablet_replica.cc:333] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.593087 10257 raft_consensus.cc:2243] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.593801 10257 raft_consensus.cc:2272] T 741facf7b8264a10bc5c3431415fc34a P b1906ab37ba541f493936685a808c8a9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.596482 10257 tablet_replica.cc:333] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9: stopping tablet replica
I20260709 13:17:36.597157 10257 raft_consensus.cc:2243] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.598186 10257 raft_consensus.cc:2272] T 3d475300e0054608a5d243d859d6609b P b1906ab37ba541f493936685a808c8a9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:37.044571 10257 tablet_server.cc:196] TabletServer@127.10.4.68:0 shutdown complete.
I20260709 13:17:37.118650 10257 master.cc:562] Master@127.10.4.126:42409 shutting down...
W20260709 13:17:37.208328 11253 catalog_manager.cc:4755] TS b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): ChangeConfig:REMOVE_PEER RPC failed for tablet 54c0e900e01b4e5aa4cf2fa0acbf6e86: Network error: Client connection negotiation failed: client connection to 127.10.4.68:37891: connect: Connection refused (error 111)
W20260709 13:17:39.480937 11253 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.68:37891: connect: Connection refused (error 111) [suppressed 237 similar messages]
W20260709 13:17:39.483855 11253 catalog_manager.cc:4755] TS b1906ab37ba541f493936685a808c8a9 (127.10.4.68:37891): ChangeConfig:REMOVE_PEER RPC failed for tablet 7a5787304dfe4697baeaeb367ee7dda3: Network error: Client connection negotiation failed: client connection to 127.10.4.68:37891: connect: Connection refused (error 111)
I20260709 13:17:40.034675 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:40.035362 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:40.035717 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P ab9dad95b38845f197fb7c3c12a492c4: stopping tablet replica
I20260709 13:17:40.291270 10257 master.cc:584] Master@127.10.4.126:42409 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (202203 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:17:40.381788 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:39849
I20260709 13:17:40.382889 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:40.388777 17850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.388897 17851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.389302 17853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.391918 10257 server_base.cc:1061] running on GCE node
I20260709 13:17:40.394722 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:40.394930 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:40.395088 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603060395071 us; error 0 us; skew 500 ppm
I20260709 13:17:40.395684 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:40.398169 10257 webserver.cc:533] Webserver started at http://127.10.4.126:38637/ using document root <none> and password file <none>
I20260709 13:17:40.398643 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:40.398806 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:40.399065 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:40.400295 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "f30b4505e27e480386540645d141f06c"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.404497 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:17:40.407810 17858 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.408483 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:40.408764 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "f30b4505e27e480386540645d141f06c"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.409019 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:40.429360 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:40.430526 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:40.467975 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:39849
I20260709 13:17:40.468066 17909 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:39849 every 8 connection(s)
I20260709 13:17:40.471868 17910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.482257 17910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c: Bootstrap starting.
I20260709 13:17:40.486702 17910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.490615 17910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c: No bootstrap required, opened a new log
I20260709 13:17:40.492694 17910 raft_consensus.cc:359] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30b4505e27e480386540645d141f06c" member_type: VOTER }
I20260709 13:17:40.493054 17910 raft_consensus.cc:385] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.493237 17910 raft_consensus.cc:740] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30b4505e27e480386540645d141f06c, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.493726 17910 consensus_queue.cc:260] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30b4505e27e480386540645d141f06c" member_type: VOTER }
I20260709 13:17:40.494112 17910 raft_consensus.cc:399] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:40.494304 17910 raft_consensus.cc:493] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:40.494503 17910 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.498553 17910 raft_consensus.cc:515] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30b4505e27e480386540645d141f06c" member_type: VOTER }
I20260709 13:17:40.498984 17910 leader_election.cc:304] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f30b4505e27e480386540645d141f06c; no voters: 
I20260709 13:17:40.499989 17910 leader_election.cc:290] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:40.500298 17913 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:40.501529 17913 raft_consensus.cc:697] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 1 LEADER]: Becoming Leader. State: Replica: f30b4505e27e480386540645d141f06c, State: Running, Role: LEADER
I20260709 13:17:40.502205 17913 consensus_queue.cc:237] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30b4505e27e480386540645d141f06c" member_type: VOTER }
I20260709 13:17:40.502718 17910 sys_catalog.cc:565] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:40.506882 17915 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f30b4505e27e480386540645d141f06c. Latest consensus state: current_term: 1 leader_uuid: "f30b4505e27e480386540645d141f06c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30b4505e27e480386540645d141f06c" member_type: VOTER } }
I20260709 13:17:40.507467 17914 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f30b4505e27e480386540645d141f06c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30b4505e27e480386540645d141f06c" member_type: VOTER } }
I20260709 13:17:40.507555 17915 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:40.507997 17914 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:40.509439 17920 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:40.514391 17920 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:40.520562 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:40.523686 17920 catalog_manager.cc:1383] Generated new cluster ID: 95a5e0d3eb39437080eea912465f2bc0
I20260709 13:17:40.524017 17920 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:40.552006 17920 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:40.553262 17920 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:40.566227 17920 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c: Generated new TSK 0
I20260709 13:17:40.566821 17920 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:40.590601 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:40.596613 17931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.597460 17932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.598766 17934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.600040 10257 server_base.cc:1061] running on GCE node
I20260709 13:17:40.601881 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:40.602072 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:40.602192 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603060602181 us; error 0 us; skew 500 ppm
I20260709 13:17:40.602675 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:40.605034 10257 webserver.cc:533] Webserver started at http://127.10.4.65:33245/ using document root <none> and password file <none>
I20260709 13:17:40.605473 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:40.605621 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:40.605840 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:40.606958 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "242d2387755e4a37ade4260e80f4eb6a"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.611482 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:40.614696 17939 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.615453 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:40.615720 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "242d2387755e4a37ade4260e80f4eb6a"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.615972 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:40.631058 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:40.632210 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:40.633651 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:40.635951 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:40.636162 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.636401 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:40.636562 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.676298 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:32893
I20260709 13:17:40.676445 18001 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:32893 every 8 connection(s)
I20260709 13:17:40.681088 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:40.688745 18006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.689635 18007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.693617 10257 server_base.cc:1061] running on GCE node
W20260709 13:17:40.694466 18010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.696115 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:40.696313 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:40.696444 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603060696430 us; error 0 us; skew 500 ppm
I20260709 13:17:40.696921 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:40.699805 10257 webserver.cc:533] Webserver started at http://127.10.4.66:32929/ using document root <none> and password file <none>
I20260709 13:17:40.700425 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:40.700682 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:40.700971 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:40.701960 18002 heartbeater.cc:344] Connected to a master server at 127.10.4.126:39849
I20260709 13:17:40.702365 18002 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:40.702383 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "fa03f4668c724e22a467d4e621f95272"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.703277 18002 heartbeater.cc:507] Master 127.10.4.126:39849 requested a full tablet report, sending...
I20260709 13:17:40.705713 17875 ts_manager.cc:194] Registered new tserver with Master: 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:40.708247 17875 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47118
I20260709 13:17:40.708379 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.002s
I20260709 13:17:40.712951 18014 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.713943 10257 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:17:40.714243 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "fa03f4668c724e22a467d4e621f95272"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.714516 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:40.728783 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:40.730001 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:40.731475 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:40.733608 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:40.733783 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.734007 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:40.734143 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.773008 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:40273
I20260709 13:17:40.773108 18076 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:40273 every 8 connection(s)
I20260709 13:17:40.777705 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:40.785043 18080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.786967 18081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.789291 18083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.790060 10257 server_base.cc:1061] running on GCE node
I20260709 13:17:40.790972 18077 heartbeater.cc:344] Connected to a master server at 127.10.4.126:39849
I20260709 13:17:40.791016 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:40.791489 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:40.791507 18077 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:40.791684 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603060791666 us; error 0 us; skew 500 ppm
I20260709 13:17:40.792397 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:40.792541 18077 heartbeater.cc:507] Master 127.10.4.126:39849 requested a full tablet report, sending...
I20260709 13:17:40.795071 17875 ts_manager.cc:194] Registered new tserver with Master: fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:40.795337 10257 webserver.cc:533] Webserver started at http://127.10.4.67:37453/ using document root <none> and password file <none>
I20260709 13:17:40.795938 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:40.796130 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:40.796406 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:40.796746 17875 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47122
I20260709 13:17:40.797679 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "1787628aa7f6489497bb871588797230"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.802932 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.002s
I20260709 13:17:40.806413 18088 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.807116 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:40.807389 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "1787628aa7f6489497bb871588797230"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.807686 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:40.837216 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:40.838433 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:40.839923 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:40.842275 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:40.842473 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.842682 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:40.842839 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.881794 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:43399
I20260709 13:17:40.881923 18150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:43399 every 8 connection(s)
I20260709 13:17:40.886914 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:40.894954 18155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.896121 18154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:40.898515 18157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.900487 18151 heartbeater.cc:344] Connected to a master server at 127.10.4.126:39849
I20260709 13:17:40.900966 18151 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:40.901842 18151 heartbeater.cc:507] Master 127.10.4.126:39849 requested a full tablet report, sending...
I20260709 13:17:40.901881 10257 server_base.cc:1061] running on GCE node
I20260709 13:17:40.902884 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:40.903120 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:40.903359 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603060903339 us; error 0 us; skew 500 ppm
I20260709 13:17:40.903968 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:40.904127 17875 ts_manager.cc:194] Registered new tserver with Master: 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:40.905844 17875 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47132
I20260709 13:17:40.906812 10257 webserver.cc:533] Webserver started at http://127.10.4.68:42131/ using document root <none> and password file <none>
I20260709 13:17:40.907707 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:40.907994 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:40.908264 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:40.909518 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-3-root/instance:
uuid: "fc8a9ec31a5745d69dae4262b271e568"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.914070 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:17:40.917690 18162 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.918506 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:40.918841 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-3-root
uuid: "fc8a9ec31a5745d69dae4262b271e568"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:17:40.919100 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:40.945429 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:40.946729 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:40.948241 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:40.950513 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:40.950695 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.950934 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:40.951077 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.990362 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.68:33171
I20260709 13:17:40.990468 18224 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.68:33171 every 8 connection(s)
I20260709 13:17:41.003597 18225 heartbeater.cc:344] Connected to a master server at 127.10.4.126:39849
I20260709 13:17:41.003993 18225 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:41.004751 18225 heartbeater.cc:507] Master 127.10.4.126:39849 requested a full tablet report, sending...
I20260709 13:17:41.006614 17875 ts_manager.cc:194] Registered new tserver with Master: fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:41.006726 10257 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.013360201s
I20260709 13:17:41.008213 17875 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47136
I20260709 13:17:41.032618 17875 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47138:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:17:41.272977 18116 tablet_service.cc:1511] Processing CreateTablet for tablet 00cc4ceb4dee449a8998229f3bd78288 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:41.274520 18116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00cc4ceb4dee449a8998229f3bd78288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.274796 18114 tablet_service.cc:1511] Processing CreateTablet for tablet d97db5bc48fc437fac40454dc3e1d871 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:41.275018 18113 tablet_service.cc:1511] Processing CreateTablet for tablet 27dcd1944f074b89a6dd10d75e8c6dd2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:41.276294 18114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97db5bc48fc437fac40454dc3e1d871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.277420 17967 tablet_service.cc:1511] Processing CreateTablet for tablet 00cc4ceb4dee449a8998229f3bd78288 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:41.278002 18115 tablet_service.cc:1511] Processing CreateTablet for tablet a7241c7eca1b4c53a4faea0bcffee354 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:41.278924 17967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00cc4ceb4dee449a8998229f3bd78288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.279309 18115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7241c7eca1b4c53a4faea0bcffee354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.280588 17965 tablet_service.cc:1511] Processing CreateTablet for tablet 6247cfa8622e49e29ad0bfba76525834 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:41.281940 17965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6247cfa8622e49e29ad0bfba76525834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.284600 17964 tablet_service.cc:1511] Processing CreateTablet for tablet 27dcd1944f074b89a6dd10d75e8c6dd2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:41.285372 17963 tablet_service.cc:1511] Processing CreateTablet for tablet 843fde44d03641ce89e8d2e247c68b2d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:41.286082 17964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27dcd1944f074b89a6dd10d75e8c6dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.286684 17963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 843fde44d03641ce89e8d2e247c68b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.288578 17966 tablet_service.cc:1511] Processing CreateTablet for tablet a7241c7eca1b4c53a4faea0bcffee354 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:41.291945 18113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27dcd1944f074b89a6dd10d75e8c6dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.289932 17966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7241c7eca1b4c53a4faea0bcffee354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.298697 18112 tablet_service.cc:1511] Processing CreateTablet for tablet 30604e2cd10b450fb3e8d52bee20f8da (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:41.300093 18112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30604e2cd10b450fb3e8d52bee20f8da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.300554 18190 tablet_service.cc:1511] Processing CreateTablet for tablet 00cc4ceb4dee449a8998229f3bd78288 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:41.301935 18190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00cc4ceb4dee449a8998229f3bd78288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.305136 18111 tablet_service.cc:1511] Processing CreateTablet for tablet 0745baef9dbd4cd0a943445a5aa9cc6b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:41.307032 18111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0745baef9dbd4cd0a943445a5aa9cc6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.308259 18110 tablet_service.cc:1511] Processing CreateTablet for tablet d4cca6423ee4465cb73609e4e335e66e (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:41.309597 18110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cca6423ee4465cb73609e4e335e66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.312599 17962 tablet_service.cc:1511] Processing CreateTablet for tablet 30604e2cd10b450fb3e8d52bee20f8da (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:41.313419 17960 tablet_service.cc:1511] Processing CreateTablet for tablet 347efb4b336e41469ded6211da2d4498 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:41.314122 17962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30604e2cd10b450fb3e8d52bee20f8da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.316530 17959 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad41548503545059d74bd2564c17b52 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:41.316911 17961 tablet_service.cc:1511] Processing CreateTablet for tablet d4cca6423ee4465cb73609e4e335e66e (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:41.317968 17959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad41548503545059d74bd2564c17b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.320971 18109 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad41548503545059d74bd2564c17b52 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:41.322403 18109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad41548503545059d74bd2564c17b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.327468 18108 tablet_service.cc:1511] Processing CreateTablet for tablet da457bda0979490fa4e5277fba14ee84 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:41.328887 18108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da457bda0979490fa4e5277fba14ee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.332994 18107 tablet_service.cc:1511] Processing CreateTablet for tablet 8989b7b66db644f89f6c4421fd51f2cc (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:41.334422 18107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8989b7b66db644f89f6c4421fd51f2cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.332994 17958 tablet_service.cc:1511] Processing CreateTablet for tablet 8989b7b66db644f89f6c4421fd51f2cc (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:41.336860 17958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8989b7b66db644f89f6c4421fd51f2cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.349596 17960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347efb4b336e41469ded6211da2d4498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.353065 17961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cca6423ee4465cb73609e4e335e66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.365077 18189 tablet_service.cc:1511] Processing CreateTablet for tablet 6247cfa8622e49e29ad0bfba76525834 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:41.366513 18189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6247cfa8622e49e29ad0bfba76525834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.368463 18183 tablet_service.cc:1511] Processing CreateTablet for tablet 3ad41548503545059d74bd2564c17b52 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:41.369858 18183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ad41548503545059d74bd2564c17b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.372576 18188 tablet_service.cc:1511] Processing CreateTablet for tablet d97db5bc48fc437fac40454dc3e1d871 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:41.373895 18188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97db5bc48fc437fac40454dc3e1d871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.374092 18116 tablet_service.cc:1511] Processing CreateTablet for tablet dd76d33e673a4966a9c91b362fafb0b1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:41.376556 18187 tablet_service.cc:1511] Processing CreateTablet for tablet 843fde44d03641ce89e8d2e247c68b2d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:41.377842 18187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 843fde44d03641ce89e8d2e247c68b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.380892 18185 tablet_service.cc:1511] Processing CreateTablet for tablet 0745baef9dbd4cd0a943445a5aa9cc6b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:41.382231 18185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0745baef9dbd4cd0a943445a5aa9cc6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.382332 17967 tablet_service.cc:1511] Processing CreateTablet for tablet a80b08582e1a4e099492c0650e820c45 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:41.383078 18111 tablet_service.cc:1511] Processing CreateTablet for tablet 1f30e4471448411bab26f97e6d34f33c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:41.384521 18184 tablet_service.cc:1511] Processing CreateTablet for tablet 347efb4b336e41469ded6211da2d4498 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:41.385273 17967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80b08582e1a4e099492c0650e820c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.385792 18184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347efb4b336e41469ded6211da2d4498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.388535 18182 tablet_service.cc:1511] Processing CreateTablet for tablet da457bda0979490fa4e5277fba14ee84 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:41.389762 18182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da457bda0979490fa4e5277fba14ee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.392491 18181 tablet_service.cc:1511] Processing CreateTablet for tablet a80b08582e1a4e099492c0650e820c45 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:41.393729 18181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80b08582e1a4e099492c0650e820c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.396281 18186 tablet_service.cc:1511] Processing CreateTablet for tablet 30604e2cd10b450fb3e8d52bee20f8da (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:41.397476 18186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30604e2cd10b450fb3e8d52bee20f8da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.375269 18116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd76d33e673a4966a9c91b362fafb0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.415464 18111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f30e4471448411bab26f97e6d34f33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.427621 17967 tablet_service.cc:1511] Processing CreateTablet for tablet dd76d33e673a4966a9c91b362fafb0b1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:41.429301 17967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd76d33e673a4966a9c91b362fafb0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.430251 17960 tablet_service.cc:1511] Processing CreateTablet for tablet 017bc4b9de4c4462845e7b860bfa6dce (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:41.431542 17960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 017bc4b9de4c4462845e7b860bfa6dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.447540 17960 tablet_service.cc:1511] Processing CreateTablet for tablet 3fd2d32e1713467387bb231dc639c10d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:41.448925 17960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fd2d32e1713467387bb231dc639c10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.463356 18107 tablet_service.cc:1511] Processing CreateTablet for tablet 017bc4b9de4c4462845e7b860bfa6dce (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:41.464421 18107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 017bc4b9de4c4462845e7b860bfa6dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.477537 18108 tablet_service.cc:1511] Processing CreateTablet for tablet 72bffd5492454e96ae4e10907b97381c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:41.484822 18245 tablet_bootstrap.cc:492] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
W20260709 13:17:41.515956 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:41.499010 18108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72bffd5492454e96ae4e10907b97381c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.531337 17964 tablet_service.cc:1511] Processing CreateTablet for tablet 127bfde2c3624e5bb2d0f68929889394 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:41.531636 18181 tablet_service.cc:1511] Processing CreateTablet for tablet 1f30e4471448411bab26f97e6d34f33c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:41.532910 17964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127bfde2c3624e5bb2d0f68929889394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.532456 18244 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:41.540310 18244 tablet_bootstrap.cc:654] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.540335 18109 tablet_service.cc:1511] Processing CreateTablet for tablet 3918cfe707c24309bd3a731cca5bcbef (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:41.490844 18188 tablet_service.cc:1511] Processing CreateTablet for tablet 3fd2d32e1713467387bb231dc639c10d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:41.543316 18040 tablet_service.cc:1511] Processing CreateTablet for tablet d97db5bc48fc437fac40454dc3e1d871 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:41.544597 18040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97db5bc48fc437fac40454dc3e1d871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.545768 18033 tablet_service.cc:1511] Processing CreateTablet for tablet 8989b7b66db644f89f6c4421fd51f2cc (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:41.547082 18033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8989b7b66db644f89f6c4421fd51f2cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.548560 18042 tablet_service.cc:1511] Processing CreateTablet for tablet a7241c7eca1b4c53a4faea0bcffee354 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:41.549844 18042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7241c7eca1b4c53a4faea0bcffee354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.551045 18037 tablet_service.cc:1511] Processing CreateTablet for tablet 0745baef9dbd4cd0a943445a5aa9cc6b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:41.552500 18037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0745baef9dbd4cd0a943445a5aa9cc6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.552331 18041 tablet_service.cc:1511] Processing CreateTablet for tablet 6247cfa8622e49e29ad0bfba76525834 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:41.553625 18041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6247cfa8622e49e29ad0bfba76525834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.554545 18036 tablet_service.cc:1511] Processing CreateTablet for tablet d4cca6423ee4465cb73609e4e335e66e (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:41.521359 18190 tablet_service.cc:1511] Processing CreateTablet for tablet dd76d33e673a4966a9c91b362fafb0b1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:41.548602 18039 tablet_service.cc:1511] Processing CreateTablet for tablet 27dcd1944f074b89a6dd10d75e8c6dd2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:41.556631 18035 tablet_service.cc:1511] Processing CreateTablet for tablet 347efb4b336e41469ded6211da2d4498 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:41.558526 18034 tablet_service.cc:1511] Processing CreateTablet for tablet da457bda0979490fa4e5277fba14ee84 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:41.558852 18187 tablet_service.cc:1511] Processing CreateTablet for tablet 72bffd5492454e96ae4e10907b97381c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:41.560768 18036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cca6423ee4465cb73609e4e335e66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.596109 18035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347efb4b336e41469ded6211da2d4498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.549760 18038 tablet_service.cc:1511] Processing CreateTablet for tablet 843fde44d03641ce89e8d2e247c68b2d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:41.599016 18038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 843fde44d03641ce89e8d2e247c68b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.564337 17958 tablet_service.cc:1511] Processing CreateTablet for tablet 72bffd5492454e96ae4e10907b97381c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:41.568521 18114 tablet_service.cc:1511] Processing CreateTablet for tablet 127bfde2c3624e5bb2d0f68929889394 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:41.576537 18109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3918cfe707c24309bd3a731cca5bcbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.579015 18115 tablet_service.cc:1511] Processing CreateTablet for tablet 848c8b98e4dd44e398eb0b2689ea69cd (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:41.580981 18112 tablet_service.cc:1511] Processing CreateTablet for tablet c65f480c3ad843578f87c43aee4ecbc6 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:41.619689 17962 tablet_service.cc:1511] Processing CreateTablet for tablet 7187dfc4efc34078a81a1f59637521c8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:41.620033 17960 tablet_service.cc:1511] Processing CreateTablet for tablet eb3ec66980be4a6ea8be1b3fa67292b8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:41.594980 18110 tablet_service.cc:1511] Processing CreateTablet for tablet 0044e0ba4e7548ea8cd3452eed875fa8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:41.593673 18185 tablet_service.cc:1511] Processing CreateTablet for tablet 3918cfe707c24309bd3a731cca5bcbef (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:41.532908 18181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f30e4471448411bab26f97e6d34f33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.622038 18116 tablet_service.cc:1511] Processing CreateTablet for tablet 7187dfc4efc34078a81a1f59637521c8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:41.622464 17967 tablet_service.cc:1511] Processing CreateTablet for tablet e1606bb8512a4c5ba5133be786298984 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:41.626749 18186 tablet_service.cc:1511] Processing CreateTablet for tablet 848c8b98e4dd44e398eb0b2689ea69cd (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:41.604552 18246 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:41.610968 18114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127bfde2c3624e5bb2d0f68929889394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.615798 17961 tablet_service.cc:1511] Processing CreateTablet for tablet 0044e0ba4e7548ea8cd3452eed875fa8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:41.616000 17966 tablet_service.cc:1511] Processing CreateTablet for tablet 848c8b98e4dd44e398eb0b2689ea69cd (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:41.617719 17963 tablet_service.cc:1511] Processing CreateTablet for tablet f0771f9326ce4105929b0fca4a301e1e (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:41.590756 18245 tablet_bootstrap.cc:654] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.628240 18185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3918cfe707c24309bd3a731cca5bcbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.603811 18034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da457bda0979490fa4e5277fba14ee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.636402 18042 tablet_service.cc:1511] Processing CreateTablet for tablet a80b08582e1a4e099492c0650e820c45 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:41.637696 18042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80b08582e1a4e099492c0650e820c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.639576 18189 tablet_service.cc:1511] Processing CreateTablet for tablet 211547823b99438c9cc10432cc919a5f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:41.640455 18115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848c8b98e4dd44e398eb0b2689ea69cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.640837 18189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211547823b99438c9cc10432cc919a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.642765 17958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72bffd5492454e96ae4e10907b97381c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.628427 18184 tablet_service.cc:1511] Processing CreateTablet for tablet c65f480c3ad843578f87c43aee4ecbc6 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:41.630255 18113 tablet_service.cc:1511] Processing CreateTablet for tablet 729ea2fca9fb40e5a0c2fd994a5adb91 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:41.650256 18112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65f480c3ad843578f87c43aee4ecbc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.651753 18247 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:41.655879 17964 tablet_service.cc:1511] Processing CreateTablet for tablet 8fee47e66a974e48926c551ee307b756 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:41.652892 18039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27dcd1944f074b89a6dd10d75e8c6dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.657554 18033 tablet_service.cc:1511] Processing CreateTablet for tablet 017bc4b9de4c4462845e7b860bfa6dce (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:41.658919 18033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 017bc4b9de4c4462845e7b860bfa6dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.631390 17959 tablet_service.cc:1511] Processing CreateTablet for tablet 211547823b99438c9cc10432cc919a5f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:41.643666 17963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0771f9326ce4105929b0fca4a301e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.665692 17966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848c8b98e4dd44e398eb0b2689ea69cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.652173 18113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 729ea2fca9fb40e5a0c2fd994a5adb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.666378 17961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0044e0ba4e7548ea8cd3452eed875fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.666661 18116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7187dfc4efc34078a81a1f59637521c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.667140 17962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7187dfc4efc34078a81a1f59637521c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.667876 17960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb3ec66980be4a6ea8be1b3fa67292b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.669922 17964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fee47e66a974e48926c551ee307b756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.649214 18190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd76d33e673a4966a9c91b362fafb0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.671522 17967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1606bb8512a4c5ba5133be786298984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.672235 18188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fd2d32e1713467387bb231dc639c10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.673111 17959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211547823b99438c9cc10432cc919a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.665303 18182 tablet_service.cc:1511] Processing CreateTablet for tablet e1606bb8512a4c5ba5133be786298984 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:41.679642 18110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0044e0ba4e7548ea8cd3452eed875fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.696322 18040 tablet_service.cc:1511] Processing CreateTablet for tablet 3fd2d32e1713467387bb231dc639c10d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:41.702756 18183 tablet_service.cc:1511] Processing CreateTablet for tablet f0771f9326ce4105929b0fca4a301e1e (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:41.704437 18041 tablet_service.cc:1511] Processing CreateTablet for tablet 1f30e4471448411bab26f97e6d34f33c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:41.706719 18246 tablet_bootstrap.cc:654] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.711480 18002 heartbeater.cc:499] Master 127.10.4.126:39849 was elected leader, sending a full tablet report...
I20260709 13:17:41.714561 18187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72bffd5492454e96ae4e10907b97381c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.727641 18111 tablet_service.cc:1511] Processing CreateTablet for tablet eb3ec66980be4a6ea8be1b3fa67292b8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:41.717684 18108 tablet_service.cc:1511] Processing CreateTablet for tablet f26de913e9ce4dad845f7173d82adfc1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:41.683264 18247 tablet_bootstrap.cc:654] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.722990 17965 tablet_service.cc:1511] Processing CreateTablet for tablet bc6bc2204cfe4f09a8f0bae7240822d8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:41.759840 18041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f30e4471448411bab26f97e6d34f33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.765373 18245 tablet_bootstrap.cc:492] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:41.765920 18245 ts_tablet_manager.cc:1403] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.281s	user 0.015s	sys 0.022s
I20260709 13:17:41.774246 18184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65f480c3ad843578f87c43aee4ecbc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.787521 18111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb3ec66980be4a6ea8be1b3fa67292b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.803465 18077 heartbeater.cc:499] Master 127.10.4.126:39849 was elected leader, sending a full tablet report...
I20260709 13:17:41.816056 18244 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:41.816596 18244 ts_tablet_manager.cc:1403] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.287s	user 0.023s	sys 0.011s
I20260709 13:17:41.817797 18116 tablet_service.cc:1511] Processing CreateTablet for tablet 8fee47e66a974e48926c551ee307b756 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:41.817929 18036 tablet_service.cc:1511] Processing CreateTablet for tablet 3918cfe707c24309bd3a731cca5bcbef (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:41.819213 18036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3918cfe707c24309bd3a731cca5bcbef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.797940 18182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1606bb8512a4c5ba5133be786298984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.830283 18247 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:41.830935 18247 ts_tablet_manager.cc:1403] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.180s	user 0.011s	sys 0.012s
I20260709 13:17:41.836459 18183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0771f9326ce4105929b0fca4a301e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.809895 18108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26de913e9ce4dad845f7173d82adfc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.838587 18181 tablet_service.cc:1511] Processing CreateTablet for tablet f26de913e9ce4dad845f7173d82adfc1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:41.840778 18181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26de913e9ce4dad845f7173d82adfc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.840451 18107 tablet_service.cc:1511] Processing CreateTablet for tablet 6200167e8cfc45a59b2b043a2601401f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:41.841114 18035 tablet_service.cc:1511] Processing CreateTablet for tablet 127bfde2c3624e5bb2d0f68929889394 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:41.841784 18107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6200167e8cfc45a59b2b043a2601401f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.843576 18186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848c8b98e4dd44e398eb0b2689ea69cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.801836 17965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6bc2204cfe4f09a8f0bae7240822d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.857489 18116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fee47e66a974e48926c551ee307b756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.858114 18244 raft_consensus.cc:359] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:41.859577 18244 raft_consensus.cc:385] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.859871 18244 raft_consensus.cc:740] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.860438 18247 raft_consensus.cc:359] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:41.860575 18244 consensus_queue.cc:260] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:41.861191 18247 raft_consensus.cc:385] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.861797 18247 raft_consensus.cc:740] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.862564 18247 consensus_queue.cc:260] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:41.864137 18245 raft_consensus.cc:359] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:41.864858 18245 raft_consensus.cc:385] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.858755 18038 tablet_service.cc:1511] Processing CreateTablet for tablet 211547823b99438c9cc10432cc919a5f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:41.873459 18115 tablet_service.cc:1511] Processing CreateTablet for tablet bc6bc2204cfe4f09a8f0bae7240822d8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:41.879806 18040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fd2d32e1713467387bb231dc639c10d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.883638 18113 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3445b301b2410eba0473bd236b619c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:41.884965 18113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3445b301b2410eba0473bd236b619c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.889544 17967 tablet_service.cc:1511] Processing CreateTablet for tablet f6de10a3b7bb4a8da2902e5f42108ed2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:41.890731 18035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127bfde2c3624e5bb2d0f68929889394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.891093 17967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6de10a3b7bb4a8da2902e5f42108ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.881022 17963 tablet_service.cc:1511] Processing CreateTablet for tablet f26de913e9ce4dad845f7173d82adfc1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:41.896755 18189 tablet_service.cc:1511] Processing CreateTablet for tablet 7187dfc4efc34078a81a1f59637521c8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:41.898168 18189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7187dfc4efc34078a81a1f59637521c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.898682 17962 tablet_service.cc:1511] Processing CreateTablet for tablet fcb98cd681e0496393f8af9091c9184b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:41.867573 18245 raft_consensus.cc:740] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.904086 18245 consensus_queue.cc:260] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:41.915244 18247 ts_tablet_manager.cc:1434] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.005s
I20260709 13:17:41.915656 18151 heartbeater.cc:499] Master 127.10.4.126:39849 was elected leader, sending a full tablet report...
I20260709 13:17:41.916515 18247 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:41.919736 18038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211547823b99438c9cc10432cc919a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.929651 18247 tablet_bootstrap.cc:654] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.931775 18115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6bc2204cfe4f09a8f0bae7240822d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.936627 18041 tablet_service.cc:1511] Processing CreateTablet for tablet c65f480c3ad843578f87c43aee4ecbc6 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:41.938215 18041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65f480c3ad843578f87c43aee4ecbc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.949395 18190 tablet_service.cc:1511] Processing CreateTablet for tablet 729ea2fca9fb40e5a0c2fd994a5adb91 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:41.967895 17959 tablet_service.cc:1511] Processing CreateTablet for tablet 0aa2530e8b624041bc011fa928d7078f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:41.974129 17964 tablet_service.cc:1511] Processing CreateTablet for tablet b7077b3a2710489282f1c1cd3f8c1ddb (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:41.983645 17963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26de913e9ce4dad845f7173d82adfc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.984457 18181 tablet_service.cc:1511] Processing CreateTablet for tablet f6de10a3b7bb4a8da2902e5f42108ed2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:41.982789 18109 tablet_service.cc:1511] Processing CreateTablet for tablet b7077b3a2710489282f1c1cd3f8c1ddb (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:41.997881 17966 tablet_service.cc:1511] Processing CreateTablet for tablet c9de6c54e6314854955dc74a83ea79d3 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:41.997881 17960 tablet_service.cc:1511] Processing CreateTablet for tablet 9f4120b61f1d44f6a4a3c5c4575f993a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:42.009210 18033 tablet_service.cc:1511] Processing CreateTablet for tablet e1606bb8512a4c5ba5133be786298984 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:42.009534 18041 tablet_service.cc:1511] Processing CreateTablet for tablet f0771f9326ce4105929b0fca4a301e1e (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:42.010620 18033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1606bb8512a4c5ba5133be786298984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.011833 18225 heartbeater.cc:499] Master 127.10.4.126:39849 was elected leader, sending a full tablet report...
I20260709 13:17:42.012668 18042 tablet_service.cc:1511] Processing CreateTablet for tablet 729ea2fca9fb40e5a0c2fd994a5adb91 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:42.014161 18042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 729ea2fca9fb40e5a0c2fd994a5adb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.017395 18041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0771f9326ce4105929b0fca4a301e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.021766 18034 tablet_service.cc:1511] Processing CreateTablet for tablet 0044e0ba4e7548ea8cd3452eed875fa8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:42.023468 18034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0044e0ba4e7548ea8cd3452eed875fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.024084 17959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa2530e8b624041bc011fa928d7078f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.025524 17960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f4120b61f1d44f6a4a3c5c4575f993a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.003101 18114 tablet_service.cc:1511] Processing CreateTablet for tablet 285967e1504c483dac972f54a3d59bfc (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 13:17:42.033568 18078 tablet.cc:2406] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:42.005043 18190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 729ea2fca9fb40e5a0c2fd994a5adb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.040294 18246 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:42.040854 18246 ts_tablet_manager.cc:1403] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.437s	user 0.008s	sys 0.017s
I20260709 13:17:42.044384 18244 ts_tablet_manager.cc:1434] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.227s	user 0.011s	sys 0.020s
I20260709 13:17:42.045553 18244 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:42.072947 18181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6de10a3b7bb4a8da2902e5f42108ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.073949 18110 tablet_service.cc:1511] Processing CreateTablet for tablet 9f4120b61f1d44f6a4a3c5c4575f993a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:42.077390 17958 tablet_service.cc:1511] Processing CreateTablet for tablet 7d07ebb0703849dbab22922f7f878958 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:42.081099 18040 tablet_service.cc:1511] Processing CreateTablet for tablet 6200167e8cfc45a59b2b043a2601401f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:42.082685 18040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6200167e8cfc45a59b2b043a2601401f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.082368 18186 tablet_service.cc:1511] Processing CreateTablet for tablet 0aa2530e8b624041bc011fa928d7078f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:42.083348 18109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7077b3a2710489282f1c1cd3f8c1ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.084108 18110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f4120b61f1d44f6a4a3c5c4575f993a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.033871 18184 tablet_service.cc:1511] Processing CreateTablet for tablet bc6bc2204cfe4f09a8f0bae7240822d8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:42.090214 18184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc6bc2204cfe4f09a8f0bae7240822d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.092573 17962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb98cd681e0496393f8af9091c9184b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.095911 17958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d07ebb0703849dbab22922f7f878958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.097604 17964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7077b3a2710489282f1c1cd3f8c1ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.100593 18188 tablet_service.cc:1511] Processing CreateTablet for tablet 6200167e8cfc45a59b2b043a2601401f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:42.102145 18188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6200167e8cfc45a59b2b043a2601401f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.104008 18186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa2530e8b624041bc011fa928d7078f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.106125 18182 tablet_service.cc:1511] Processing CreateTablet for tablet b7077b3a2710489282f1c1cd3f8c1ddb (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:42.112095 18039 tablet_service.cc:1511] Processing CreateTablet for tablet 8fee47e66a974e48926c551ee307b756 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:42.113687 18252 raft_consensus.cc:493] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.114272 18252 raft_consensus.cc:515] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:42.117012 18247 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:42.117556 18247 ts_tablet_manager.cc:1403] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.201s	user 0.026s	sys 0.035s
I20260709 13:17:41.987493 18108 tablet_service.cc:1511] Processing CreateTablet for tablet fcb98cd681e0496393f8af9091c9184b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:42.144474 18108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb98cd681e0496393f8af9091c9184b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.148330 18111 tablet_service.cc:1511] Processing CreateTablet for tablet 7d07ebb0703849dbab22922f7f878958 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:42.147500 17966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9de6c54e6314854955dc74a83ea79d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.149914 18111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d07ebb0703849dbab22922f7f878958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.153488 18035 tablet_service.cc:1511] Processing CreateTablet for tablet f6de10a3b7bb4a8da2902e5f42108ed2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:42.155028 18035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6de10a3b7bb4a8da2902e5f42108ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.156411 18245 ts_tablet_manager.cc:1434] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.390s	user 0.013s	sys 0.008s
I20260709 13:17:42.158499 18037 tablet_service.cc:1511] Processing CreateTablet for tablet eb3ec66980be4a6ea8be1b3fa67292b8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:42.159229 18247 raft_consensus.cc:359] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:42.160277 18247 raft_consensus.cc:385] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.160609 18247 raft_consensus.cc:740] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.142879 18042 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3445b301b2410eba0473bd236b619c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:42.163722 18246 raft_consensus.cc:359] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:42.178023 18038 tablet_service.cc:1511] Processing CreateTablet for tablet 285967e1504c483dac972f54a3d59bfc (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:42.108343 18185 tablet_service.cc:1511] Processing CreateTablet for tablet 285967e1504c483dac972f54a3d59bfc (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:42.181193 18247 consensus_queue.cc:260] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:42.182731 18113 tablet_service.cc:1511] Processing CreateTablet for tablet f1ef84f551b343d3ad57cf6aa39cf47f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:42.184643 18114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 285967e1504c483dac972f54a3d59bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.185668 18113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ef84f551b343d3ad57cf6aa39cf47f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.191577 18247 ts_tablet_manager.cc:1434] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.074s	user 0.009s	sys 0.000s
I20260709 13:17:42.192268 18245 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:42.180379 18246 raft_consensus.cc:385] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.196307 18182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7077b3a2710489282f1c1cd3f8c1ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.193333 18252 leader_election.cc:290] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:42.213284 18187 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3445b301b2410eba0473bd236b619c (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:42.205682 18247 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:42.209513 18039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fee47e66a974e48926c551ee307b756. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:42.218648 17860 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 76801.9us
I20260709 13:17:42.227972 18254 raft_consensus.cc:493] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.228571 18254 raft_consensus.cc:515] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:42.207716 18246 raft_consensus.cc:740] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.175094 18036 tablet_service.cc:1511] Processing CreateTablet for tablet 0aa2530e8b624041bc011fa928d7078f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:42.246186 18042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3445b301b2410eba0473bd236b619c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.238722 18041 tablet_service.cc:1511] Processing CreateTablet for tablet fcb98cd681e0496393f8af9091c9184b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:42.244220 18185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 285967e1504c483dac972f54a3d59bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.247941 18038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 285967e1504c483dac972f54a3d59bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.268595 18183 tablet_service.cc:1511] Processing CreateTablet for tablet 7d07ebb0703849dbab22922f7f878958 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:42.267060 18189 tablet_service.cc:1511] Processing CreateTablet for tablet c9de6c54e6314854955dc74a83ea79d3 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:42.280169 18187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3445b301b2410eba0473bd236b619c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.267264 18246 consensus_queue.cc:260] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:42.265455 18254 leader_election.cc:290] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:42.291908 18189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9de6c54e6314854955dc74a83ea79d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.294203 18245 tablet_bootstrap.cc:654] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.297580 18037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb3ec66980be4a6ea8be1b3fa67292b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.305047 18183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d07ebb0703849dbab22922f7f878958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.328390 18244 tablet_bootstrap.cc:654] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.337046 18253 raft_consensus.cc:493] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.337592 18253 raft_consensus.cc:515] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:42.339993 18246 ts_tablet_manager.cc:1434] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.299s	user 0.013s	sys 0.000s
I20260709 13:17:42.340230 18253 leader_election.cc:290] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:42.344244 18246 tablet_bootstrap.cc:492] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:42.344928 18034 tablet_service.cc:1511] Processing CreateTablet for tablet c9de6c54e6314854955dc74a83ea79d3 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:42.346143 18041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb98cd681e0496393f8af9091c9184b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.391696 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "843fde44d03641ce89e8d2e247c68b2d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:42.392675 17862 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 140413us
I20260709 13:17:42.395927 18040 tablet_service.cc:1511] Processing CreateTablet for tablet 9f4120b61f1d44f6a4a3c5c4575f993a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:17:42.395982 17859 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 47993.2us
I20260709 13:17:42.348645 18033 tablet_service.cc:1511] Processing CreateTablet for tablet f1ef84f551b343d3ad57cf6aa39cf47f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:42.343281 18181 tablet_service.cc:1511] Processing CreateTablet for tablet f1ef84f551b343d3ad57cf6aa39cf47f (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:42.400430 18181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ef84f551b343d3ad57cf6aa39cf47f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:42.401537 17940 leader_election.cc:343] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:42.373692 18036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa2530e8b624041bc011fa928d7078f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.410641 18033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ef84f551b343d3ad57cf6aa39cf47f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.427443 18034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9de6c54e6314854955dc74a83ea79d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.439728 18265 raft_consensus.cc:493] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.440423 18265 raft_consensus.cc:515] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:42.461679 18245 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:42.462296 18245 ts_tablet_manager.cc:1403] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.270s	user 0.008s	sys 0.008s
I20260709 13:17:42.465382 18245 raft_consensus.cc:359] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:42.470997 18245 raft_consensus.cc:385] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.471330 18245 raft_consensus.cc:740] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.472187 18245 consensus_queue.cc:260] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:42.474615 18245 ts_tablet_manager.cc:1434] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:17:42.475698 18245 tablet_bootstrap.cc:492] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:42.477082 18246 tablet_bootstrap.cc:654] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.478430 18247 tablet_bootstrap.cc:654] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.495064 18040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f4120b61f1d44f6a4a3c5c4575f993a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.501016 18244 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:42.501601 18244 ts_tablet_manager.cc:1403] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.456s	user 0.010s	sys 0.011s
I20260709 13:17:42.448978 18265 leader_election.cc:290] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:42.508065 18244 raft_consensus.cc:359] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:42.508900 18244 raft_consensus.cc:385] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:42.596055 18245 tablet_bootstrap.cc:700] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230: Time spent opening tablet: real 0.116s	user 0.002s	sys 0.000s
I20260709 13:17:42.597594 18244 raft_consensus.cc:740] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:42.614141 18245 tablet_bootstrap.cc:654] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:42.620059 18244 consensus_queue.cc:260] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:42.622211 18244 ts_tablet_manager.cc:1434] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.120s	user 0.007s	sys 0.000s
I20260709 13:17:42.623926 18244 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:42.629876 18244 tablet_bootstrap.cc:654] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:42.671794 17963 consensus_meta.cc:287] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.011s
I20260709 13:17:42.674634 17963 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55470 (request call id 23) took 1053 ms. Trace:
I20260709 13:17:42.675359 17963 rpcz_store.cc:276] 0709 13:17:41.621450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.798734 (+177284us) service_pool.cc:224] Handling call
0709 13:17:41.893386 (+ 94652us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:41.893487 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.169096 (+275609us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:42.673238 (+504142us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:42.674407 (+  1169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:42.681061 18246 tablet_bootstrap.cc:492] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:42.682333 18246 ts_tablet_manager.cc:1403] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.338s	user 0.023s	sys 0.007s
I20260709 13:17:42.685325 18254 raft_consensus.cc:493] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:42.714679 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:43.692243 17998 debug-util.cc:398] Leaking SignalData structure 0x7b0800196c40 after lost signal to thread 10260
W20260709 13:17:43.732075 10262 debug-util.cc:398] Leaking SignalData structure 0x7b08003b7ac0 after lost signal to thread 18018
W20260709 13:17:43.732695 10262 kernel_stack_watchdog.cc:198] Thread 18018 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 372ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260709 13:17:43.736537 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:43.819058 18036 consensus_meta.cc:287] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272: Time spent flushing consensus metadata: real 1.329s	user 0.020s	sys 0.015s
W20260709 13:17:43.864611 17962 consensus_meta.cc:287] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Time spent flushing consensus metadata: real 1.590s	user 0.020s	sys 0.028s
I20260709 13:17:42.686579 18254 raft_consensus.cc:515] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:43.866326 17962 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55470 (request call id 27) took 2148 ms. Trace:
I20260709 13:17:43.866827 17962 rpcz_store.cc:276] 0709 13:17:41.718163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.897840 (+179677us) service_pool.cc:224] Handling call
0709 13:17:41.954628 (+ 56788us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:41.959222 (+  4594us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.274720 (+315498us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:43.865406 (+1590686us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:43.866244 (+   838us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:42.689774 18246 raft_consensus.cc:359] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:43.868989 18254 leader_election.cc:290] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:43.869321 18246 raft_consensus.cc:385] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:43.869721 18246 raft_consensus.cc:740] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:43.870501 18246 consensus_queue.cc:260] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:43.873404 18246 ts_tablet_manager.cc:1434] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 1.191s	user 0.010s	sys 0.019s
W20260709 13:17:43.873740 18246 ts_tablet_manager.cc:1467] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: Tablet startup took 1191ms
W20260709 13:17:43.874078 18246 ts_tablet_manager.cc:1469] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: Trace:
0709 13:17:41.364391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.371502 (+  7111us) service_pool.cc:224] Handling call
0709 13:17:41.373405 (+  1903us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:41.373489 (+    84us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:41.472533 (+ 99044us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:41.488856 (+ 16323us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:41.489391 (+   535us) inbound_call.cc:177] Queueing success response
0709 13:17:42.343901 (+854510us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:17:42.682855 (+338954us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:17:42.684247 (+  1392us) tablet_replica.cc:261] Starting instrumentation
0709 13:17:42.685202 (+   955us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":809,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":166,"tablet-open.queue_time_us":854615,"thread_start_us":472,"threads_started":1}
I20260709 13:17:43.877354 18245 tablet_bootstrap.cc:492] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:43.877539 18246 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:43.878176 18245 ts_tablet_manager.cc:1403] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 1.403s	user 0.025s	sys 0.011s
I20260709 13:17:43.882910 18245 raft_consensus.cc:359] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:43.883791 18245 raft_consensus.cc:385] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:43.884119 18245 raft_consensus.cc:740] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:43.884935 18245 consensus_queue.cc:260] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:43.886837 18245 ts_tablet_manager.cc:1434] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:17:43.887148 18246 tablet_bootstrap.cc:654] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:43.888008 18245 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:43.893147 18246 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:43.893697 18246 ts_tablet_manager.cc:1403] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:17:43.895712 18245 tablet_bootstrap.cc:654] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:43.896808 18246 raft_consensus.cc:359] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:43.897588 18246 raft_consensus.cc:385] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:43.897855 18246 raft_consensus.cc:740] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:43.898571 18246 consensus_queue.cc:260] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:43.901567 18246 ts_tablet_manager.cc:1434] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:43.909292 18246 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:43.917042 18246 tablet_bootstrap.cc:654] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:44.073097 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:44.075439 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:44.075874 18017 leader_election.cc:343] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.077732 18256 raft_consensus.cc:493] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:44.078313 18018 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 1.719s	user 0.052s	sys 0.091s
I20260709 13:17:44.078562 18256 raft_consensus.cc:515] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:44.080013 18114 consensus_meta.cc:287] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: Time spent flushing consensus metadata: real 1.770s	user 0.021s	sys 0.016s
I20260709 13:17:44.081750 18114 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:39874 (request call id 29) took 2309 ms. Trace:
I20260709 13:17:44.083535 18114 rpcz_store.cc:276] 0709 13:17:41.780113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.002089 (+221976us) service_pool.cc:224] Handling call
0709 13:17:42.156625 (+154536us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b5800108998. stack: 00007f35b98df343 00007f35b98df240 00007f35b926e1d4 00007f35c2fac304 00007f35c2fac299 00007f35b9799db1 00007f35b98d9b71 00007f35b98d9a22 00007f35c27f2c09 00007f35c194cee0 00007f35c1a32831 00007f35c1a0fac3 00007f35c19be400 00007f35bae43af4 00007f35bae43a82 00007f35bae439b1
0709 13:17:42.180491 (+ 23866us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.184272 (+  3781us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.309461 (+125189us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:44.080895 (+1771434us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.081630 (+   735us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26681472}
I20260709 13:17:44.094136 18256 leader_election.cc:290] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
W20260709 13:17:44.085389 18018 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 1.726s	user 0.052s	sys 0.092s
I20260709 13:17:44.098888 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:44.087006 18165 leader_election.cc:343] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:44.104367 18018 leader_election.cc:343] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.104877 18018 leader_election.cc:304] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:44.116380 18245 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:44.117085 18245 ts_tablet_manager.cc:1403] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.229s	user 0.003s	sys 0.008s
W20260709 13:17:44.121356 18090 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 117646us
W20260709 13:17:44.124246 18041 consensus_meta.cc:287] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272: Time spent flushing consensus metadata: real 1.714s	user 0.027s	sys 0.011s
W20260709 13:17:44.107137 18034 consensus_meta.cc:287] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272: Time spent flushing consensus metadata: real 1.605s	user 0.020s	sys 0.026s
I20260709 13:17:44.128031 18245 raft_consensus.cc:359] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:44.128525 18244 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:44.129320 18244 ts_tablet_manager.cc:1403] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 1.506s	user 0.018s	sys 0.060s
I20260709 13:17:44.128840 18245 raft_consensus.cc:385] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:44.130095 18245 raft_consensus.cc:740] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:44.130949 18245 consensus_queue.cc:260] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:44.133883 18245 ts_tablet_manager.cc:1434] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:17:44.133749 18244 raft_consensus.cc:359] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:44.134850 18244 raft_consensus.cc:385] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:44.135251 18244 raft_consensus.cc:740] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:44.135592 18245 tablet_bootstrap.cc:492] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
W20260709 13:17:44.135922 17966 consensus_meta.cc:287] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Time spent flushing consensus metadata: real 1.883s	user 0.007s	sys 0.023s
I20260709 13:17:44.136200 18244 consensus_queue.cc:260] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:44.140368 18253 raft_consensus.cc:493] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.141299 18244 ts_tablet_manager.cc:1434] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:17:44.141050 18253 raft_consensus.cc:515] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:44.142901 18244 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:44.144646 18288 raft_consensus.cc:2749] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:44.128535 18246 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:44.146368 18246 ts_tablet_manager.cc:1403] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.244s	user 0.080s	sys 0.073s
I20260709 13:17:44.147063 18253 leader_election.cc:290] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:44.147528 18037 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 21) took 2587 ms. Trace:
I20260709 13:17:44.147974 18037 rpcz_store.cc:276] 0709 13:17:41.560371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.070630 (+510259us) service_pool.cc:224] Handling call
0709 13:17:42.159532 (+ 88902us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.159653 (+   121us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.435545 (+275892us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:42.494683 (+ 59138us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.147212 (+1652529us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.149835 18246 raft_consensus.cc:359] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:44.150398 18037 tablet_service.cc:1511] Processing CreateTablet for tablet b8f2c37906b94d31b85748436677deef (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:17:44.156652 18246 raft_consensus.cc:385] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:44.157069 18246 raft_consensus.cc:740] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:44.157948 18246 consensus_queue.cc:260] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:44.158506 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "843fde44d03641ce89e8d2e247c68b2d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.160045 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.160594 18246 ts_tablet_manager.cc:1434] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:17:44.162124 18246 tablet_bootstrap.cc:492] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:44.166885 18189 tablet_service.cc:1511] Processing CreateTablet for tablet 15a02bb6e41942bba7f6cc5358ef1506 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:44.167471 18181 tablet_service.cc:1511] Processing CreateTablet for tablet b8f2c37906b94d31b85748436677deef (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:17:44.168685 18189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15a02bb6e41942bba7f6cc5358ef1506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.155586 18244 tablet_bootstrap.cc:654] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:44.188328 17964 consensus_meta.cc:287] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Time spent flushing consensus metadata: real 1.999s	user 0.048s	sys 0.010s
I20260709 13:17:44.191383 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:44.197368 18200 raft_consensus.cc:2468] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:44.197993 17964 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55470 (request call id 29) took 2419 ms. Trace:
I20260709 13:17:44.198426 17964 rpcz_store.cc:276] 0709 13:17:41.783286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.973222 (+189936us) service_pool.cc:224] Handling call
0709 13:17:41.974981 (+  1759us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:41.975069 (+    88us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.189330 (+214261us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:44.197259 (+2007929us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.197925 (+   666us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.200196 18190 tablet_service.cc:1511] Processing CreateTablet for tablet eb77b44e31244ec29dad965bc258249a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:44.201508 18190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb77b44e31244ec29dad965bc258249a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.170830 18246 tablet_bootstrap.cc:654] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:44.202981 17998 debug-util.cc:398] Leaking SignalData structure 0x7b0800167f80 after lost signal to thread 17909
I20260709 13:17:44.184641 18182 tablet_service.cc:1511] Processing CreateTablet for tablet a4c004aeb9af431087cd2e10becd16c2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:44.204932 18182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c004aeb9af431087cd2e10becd16c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.207903 18035 tablet_service.cc:1511] Processing CreateTablet for tablet 020d6114c75749cfab1d9c0b047da996 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:44.186298 18187 tablet_service.cc:1511] Processing CreateTablet for tablet 1c67ac9ecbb841cea07be431d6a78d54 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:44.209792 18181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f2c37906b94d31b85748436677deef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.187565 18186 tablet_service.cc:1511] Processing CreateTablet for tablet b2861a57f9034f7fa50b7ff5bd98789b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:44.211627 18186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2861a57f9034f7fa50b7ff5bd98789b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.190470 17966 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55470 (request call id 31) took 2226 ms. Trace:
I20260709 13:17:44.212414 17966 rpcz_store.cc:276] 0709 13:17:41.964250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.996951 (+ 32701us) service_pool.cc:224] Handling call
0709 13:17:41.999797 (+  2846us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.000200 (+   403us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.234928 (+234728us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:44.189768 (+1954840us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.190398 (+   630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.169817 18183 tablet_service.cc:1511] Processing CreateTablet for tablet 110dca2dfd484809b9e058a7ab22b49a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:44.216795 18183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 110dca2dfd484809b9e058a7ab22b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.217521 18187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c67ac9ecbb841cea07be431d6a78d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.178673 18188 tablet_service.cc:1511] Processing CreateTablet for tablet 691cfc20fdd444b3b53db6fdcdedcda8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:44.219344 18188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691cfc20fdd444b3b53db6fdcdedcda8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.154356 18245 tablet_bootstrap.cc:654] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:44.224155 18189 tablet_service.cc:1511] Processing CreateTablet for tablet d4bd29ab75004ddf9b1ce255a071ed24 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:44.228703 18184 tablet_service.cc:1511] Processing CreateTablet for tablet 79c18e4eea964029ad14dc02ce2137e2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:44.229743 18185 tablet_service.cc:1511] Processing CreateTablet for tablet c143c20585e74a46807a580f6dcdafc1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:44.230255 18184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c18e4eea964029ad14dc02ce2137e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.234429 18189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bd29ab75004ddf9b1ce255a071ed24. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:44.236358 17998 debug-util.cc:398] Leaking SignalData structure 0x7b08001e8c80 after lost signal to thread 18039
I20260709 13:17:44.237026 18038 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 29) took 2477 ms. Trace:
W20260709 13:17:44.237471 17998 debug-util.cc:398] Leaking SignalData structure 0x7b0800050620 after lost signal to thread 18076
W20260709 13:17:44.238003 17998 debug-util.cc:398] Leaking SignalData structure 0x7b08002a3920 after lost signal to thread 18077
I20260709 13:17:44.239126 18039 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 24) took 2677 ms. Trace:
I20260709 13:17:44.239672 18039 rpcz_store.cc:276] 0709 13:17:41.561300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.111222 (+549922us) service_pool.cc:224] Handling call
0709 13:17:42.126517 (+ 15295us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.161963 (+ 35446us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.324587 (+162624us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:42.504056 (+179469us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.238896 (+1734840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.243988 18042 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 26) took 2622 ms. Trace:
I20260709 13:17:44.244419 18042 rpcz_store.cc:276] 0709 13:17:41.621502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.141783 (+520281us) service_pool.cc:224] Handling call
0709 13:17:42.164068 (+ 22285us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.164178 (+   110us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.704463 (+540285us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:43.163243 (+458780us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.243792 (+1080549us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.244993 18039 tablet_service.cc:1511] Processing CreateTablet for tablet 691cfc20fdd444b3b53db6fdcdedcda8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:44.246196 18042 tablet_service.cc:1511] Processing CreateTablet for tablet 46d0d9f275f44ae59f4f6c2df48acd6d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 13:17:44.248525 18018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 144208us
I20260709 13:17:44.244024 18036 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 28) took 2517 ms. Trace:
I20260709 13:17:44.246646 18039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691cfc20fdd444b3b53db6fdcdedcda8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.249473 18041 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 27) took 2530 ms. Trace:
I20260709 13:17:44.256273 18280 raft_consensus.cc:493] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.250710 18036 rpcz_store.cc:276] 0709 13:17:41.726416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.174042 (+447626us) service_pool.cc:224] Handling call
0709 13:17:42.250424 (+ 76382us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.250531 (+   107us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.489696 (+239165us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:43.819830 (+1330134us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.243965 (+424135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.258093 18040 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 30) took 2443 ms. Trace:
I20260709 13:17:44.258378 18036 tablet_service.cc:1511] Processing CreateTablet for tablet 110dca2dfd484809b9e058a7ab22b49a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:44.237427 18038 rpcz_store.cc:276] 0709 13:17:41.759483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.177143 (+417660us) service_pool.cc:224] Handling call
0709 13:17:42.200831 (+ 23688us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.207942 (+  7111us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.432213 (+224271us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:42.505608 (+ 73395us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.236835 (+1731227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.260138 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:44.260787 18038 tablet_service.cc:1511] Processing CreateTablet for tablet 15a02bb6e41942bba7f6cc5358ef1506 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:44.253697 18185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c143c20585e74a46807a580f6dcdafc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.256824 18280 raft_consensus.cc:515] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.277268 18280 leader_election.cc:290] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:44.414341 17966 tablet_service.cc:1511] Processing CreateTablet for tablet 020d6114c75749cfab1d9c0b047da996 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:44.415851 17966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 020d6114c75749cfab1d9c0b047da996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.416013 17964 tablet_service.cc:1511] Processing CreateTablet for tablet b8f2c37906b94d31b85748436677deef (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:17:44.417362 17964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f2c37906b94d31b85748436677deef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.424705 17958 tablet_service.cc:1511] Processing CreateTablet for tablet 46d0d9f275f44ae59f4f6c2df48acd6d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:44.425567 17963 tablet_service.cc:1511] Processing CreateTablet for tablet 110dca2dfd484809b9e058a7ab22b49a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:44.426185 17958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46d0d9f275f44ae59f4f6c2df48acd6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.426824 17963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 110dca2dfd484809b9e058a7ab22b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.428671 17967 tablet_service.cc:1511] Processing CreateTablet for tablet 1c67ac9ecbb841cea07be431d6a78d54 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:44.430020 17967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c67ac9ecbb841cea07be431d6a78d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.436281 18111 tablet_service.cc:1511] Processing CreateTablet for tablet 691cfc20fdd444b3b53db6fdcdedcda8 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:44.437866 18111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 691cfc20fdd444b3b53db6fdcdedcda8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.437958 18114 tablet_service.cc:1511] Processing CreateTablet for tablet 020d6114c75749cfab1d9c0b047da996 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:44.439160 18252 raft_consensus.cc:493] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.439685 18252 raft_consensus.cc:515] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:44.441540 17965 tablet_service.cc:1511] Processing CreateTablet for tablet 6165069b9c0d4a2bb1637ab3ade3fe09 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:44.442272 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:44.442946 17965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6165069b9c0d4a2bb1637ab3ade3fe09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.436199 17940 leader_election.cc:304] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
W20260709 13:17:44.448468 17998 debug-util.cc:398] Leaking SignalData structure 0x7b0800333080 after lost signal to thread 18151
I20260709 13:17:44.449538 18252 leader_election.cc:290] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
W20260709 13:17:44.449855 17998 debug-util.cc:398] Leaking SignalData structure 0x7b08002c3b00 after lost signal to thread 18224
I20260709 13:17:44.451583 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30604e2cd10b450fb3e8d52bee20f8da" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:44.452486 17966 tablet_service.cc:1511] Processing CreateTablet for tablet 4bd55fcdd2224c1db74729d68acc0ee6 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:44.453806 17966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bd55fcdd2224c1db74729d68acc0ee6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:44.454175 17940 leader_election.cc:343] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.454763 18252 raft_consensus.cc:2804] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:44.440220 18034 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 31) took 2475 ms. Trace:
I20260709 13:17:44.459666 18252 raft_consensus.cc:493] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.460758 18252 raft_consensus.cc:3060] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.462845 18037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f2c37906b94d31b85748436677deef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.467662 18114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 020d6114c75749cfab1d9c0b047da996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.467777 18036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 110dca2dfd484809b9e058a7ab22b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.473819 18112 tablet_service.cc:1511] Processing CreateTablet for tablet 4bd55fcdd2224c1db74729d68acc0ee6 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:44.475373 18112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bd55fcdd2224c1db74729d68acc0ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.477296 18109 tablet_service.cc:1511] Processing CreateTablet for tablet eb77b44e31244ec29dad965bc258249a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:44.478706 18109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb77b44e31244ec29dad965bc258249a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.474470 18252 raft_consensus.cc:515] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.473565 18038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15a02bb6e41942bba7f6cc5358ef1506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.490607 18252 leader_election.cc:290] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:44.506781 18035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 020d6114c75749cfab1d9c0b047da996. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.509325 18110 tablet_service.cc:1511] Processing CreateTablet for tablet c143c20585e74a46807a580f6dcdafc1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:44.510787 18110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c143c20585e74a46807a580f6dcdafc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.512540 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:44.513214 18200 raft_consensus.cc:3060] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.476557 18108 tablet_service.cc:1511] Processing CreateTablet for tablet 1c67ac9ecbb841cea07be431d6a78d54 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:44.517079 18108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c67ac9ecbb841cea07be431d6a78d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.517858 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:44.526664 18116 tablet_service.cc:1511] Processing CreateTablet for tablet 6165069b9c0d4a2bb1637ab3ade3fe09 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:44.536901 18116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6165069b9c0d4a2bb1637ab3ade3fe09. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:44.541604 18017 leader_election.cc:343] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:44.545830 17941 leader_election.cc:343] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.546123 18245 tablet_bootstrap.cc:492] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:44.546736 18245 ts_tablet_manager.cc:1403] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.412s	user 0.014s	sys 0.015s
I20260709 13:17:44.547726 17959 tablet_service.cc:1511] Processing CreateTablet for tablet b2861a57f9034f7fa50b7ff5bd98789b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:44.549106 17959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2861a57f9034f7fa50b7ff5bd98789b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.549966 18245 raft_consensus.cc:359] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.550738 18245 raft_consensus.cc:385] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:44.551003 18245 raft_consensus.cc:740] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:44.460325 18034 rpcz_store.cc:276] 0709 13:17:41.965074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.344085 (+379011us) service_pool.cc:224] Handling call
0709 13:17:42.345740 (+  1655us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.345837 (+    97us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.501717 (+155880us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:44.125944 (+1624227us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.440141 (+314197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.552560 18113 tablet_service.cc:1511] Processing CreateTablet for tablet 46d0d9f275f44ae59f4f6c2df48acd6d (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:44.553247 18034 tablet_service.cc:1511] Processing CreateTablet for tablet 4bd55fcdd2224c1db74729d68acc0ee6 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:44.553889 18113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46d0d9f275f44ae59f4f6c2df48acd6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.557348 17941 leader_election.cc:304] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:44.559285 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:44.560118 17976 raft_consensus.cc:2393] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 1: Already voted for candidate 242d2387755e4a37ade4260e80f4eb6a in this term.
I20260709 13:17:44.560856 18252 raft_consensus.cc:2749] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:44.258512 18040 rpcz_store.cc:276] 0709 13:17:41.814675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.343200 (+528525us) service_pool.cc:224] Handling call
0709 13:17:42.396897 (+ 53697us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.396998 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:44.093568 (+1696570us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:44.101905 (+  8337us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.257933 (+156028us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.573272 17961 tablet_service.cc:1511] Processing CreateTablet for tablet d4bd29ab75004ddf9b1ce255a071ed24 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:44.574751 17961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bd29ab75004ddf9b1ce255a071ed24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.576181 18091 leader_election.cc:304] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:44.577010 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:44.577445 18254 raft_consensus.cc:3060] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.577705 18199 raft_consensus.cc:2468] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:44.579385 18089 leader_election.cc:304] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:44.580458 18280 raft_consensus.cc:493] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.581288 18280 raft_consensus.cc:515] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:44.583580 18280 leader_election.cc:290] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:44.563159 17960 tablet_service.cc:1511] Processing CreateTablet for tablet a4c004aeb9af431087cd2e10becd16c2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:44.585290 17960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c004aeb9af431087cd2e10becd16c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.526628 18107 tablet_service.cc:1511] Processing CreateTablet for tablet 15a02bb6e41942bba7f6cc5358ef1506 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:44.580523 17966 tablet_service.cc:1511] Processing CreateTablet for tablet 79c18e4eea964029ad14dc02ce2137e2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:44.586220 18037 tablet_service.cc:1511] Processing CreateTablet for tablet eb77b44e31244ec29dad965bc258249a (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:44.489050 18247 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:44.570873 18040 tablet_service.cc:1511] Processing CreateTablet for tablet a4c004aeb9af431087cd2e10becd16c2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:44.601154 18246 tablet_bootstrap.cc:492] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:44.604000 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "843fde44d03641ce89e8d2e247c68b2d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:44.621152 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.621683 17976 raft_consensus.cc:2468] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:44.611088 18247 ts_tablet_manager.cc:1403] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 2.419s	user 0.032s	sys 0.033s
I20260709 13:17:44.618150 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.621280 18246 ts_tablet_manager.cc:1403] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.460s	user 0.012s	sys 0.032s
I20260709 13:17:44.626220 18165 leader_election.cc:304] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:44.627322 18280 raft_consensus.cc:2804] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:44.627709 18280 raft_consensus.cc:493] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.616236 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.628144 17962 tablet_service.cc:1511] Processing CreateTablet for tablet c143c20585e74a46807a580f6dcdafc1 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 13:17:44.617594 18018 leader_election.cc:343] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.628715 18246 raft_consensus.cc:359] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.623006 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "843fde44d03641ce89e8d2e247c68b2d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.629025 18018 leader_election.cc:304] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:44.617203 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.628026 18280 raft_consensus.cc:3060] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.554513 18034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bd55fcdd2224c1db74729d68acc0ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.600859 17966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c18e4eea964029ad14dc02ce2137e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.629490 18246 raft_consensus.cc:385] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:44.635752 18253 raft_consensus.cc:2749] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:44.638000 18090 leader_election.cc:343] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:44.638953 18164 leader_election.cc:343] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:44.639657 18164 leader_election.cc:343] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.640044 18164 leader_election.cc:304] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:44.253429 18041 rpcz_store.cc:276] 0709 13:17:41.719289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.170794 (+451505us) service_pool.cc:224] Handling call
0709 13:17:42.251194 (+ 80400us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.251288 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:42.409506 (+158218us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:44.125116 (+1715610us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.249412 (+124296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:44.642675 18041 tablet_service.cc:1511] Processing CreateTablet for tablet 6165069b9c0d4a2bb1637ab3ade3fe09 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:44.643260 17874 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47122 (request call id 5) took 2321 ms. Trace:
I20260709 13:17:44.643666 17874 rpcz_store.cc:276] 0709 13:17:42.321343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.771091 (+449748us) service_pool.cc:224] Handling call
0709 13:17:44.643064 (+1871973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52279,"reported_tablets":1}
I20260709 13:17:44.644240 18107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15a02bb6e41942bba7f6cc5358ef1506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.645447 18115 tablet_service.cc:1511] Processing CreateTablet for tablet b2861a57f9034f7fa50b7ff5bd98789b (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:44.645632 18036 tablet_service.cc:1511] Processing CreateTablet for tablet d4bd29ab75004ddf9b1ce255a071ed24 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:44.646750 18115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2861a57f9034f7fa50b7ff5bd98789b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.663816 17962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c143c20585e74a46807a580f6dcdafc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.665153 18244 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:44.665757 18244 ts_tablet_manager.cc:1403] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.523s	user 0.019s	sys 0.034s
I20260709 13:17:44.667598 18041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6165069b9c0d4a2bb1637ab3ade3fe09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.668540 18036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bd29ab75004ddf9b1ce255a071ed24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.669508 18042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46d0d9f275f44ae59f4f6c2df48acd6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.680699 18109 tablet_service.cc:1511] Processing CreateTablet for tablet 79c18e4eea964029ad14dc02ce2137e2 (DEFAULT_TABLE table=test-workload [id=5c3228aaeeca4e6f823c5e801dcf20a9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:44.625389 18247 raft_consensus.cc:359] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:44.682214 18109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c18e4eea964029ad14dc02ce2137e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.682157 18247 raft_consensus.cc:385] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:44.695796 18247 raft_consensus.cc:740] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:44.696791 18247 consensus_queue.cc:260] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:44.698541 18244 raft_consensus.cc:359] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:44.634895 18246 raft_consensus.cc:740] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:44.700016 18040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c004aeb9af431087cd2e10becd16c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.700217 18246 consensus_queue.cc:260] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.699182 18247 ts_tablet_manager.cc:1434] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.000s
I20260709 13:17:44.713004 18033 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44092 (request call id 32) took 2595 ms. Trace:
I20260709 13:17:44.713557 18033 rpcz_store.cc:276] 0709 13:17:42.117135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.347750 (+230615us) service_pool.cc:224] Handling call
0709 13:17:42.398676 (+ 50926us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:42.398778 (+   102us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:44.502565 (+2103787us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:44.712325 (+209760us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:44.712842 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:44.717397 17862 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 280347us
I20260709 13:17:44.719822 17873 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47132 (request call id 2) took 2742 ms. Trace:
I20260709 13:17:44.720283 17873 rpcz_store.cc:276] 0709 13:17:41.977340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.978301 (+   961us) service_pool.cc:224] Handling call
0709 13:17:44.719716 (+2741415us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88157,"reported_tablets":17}
W20260709 13:17:44.723905 17859 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 166586us
I20260709 13:17:44.723603 18245 consensus_queue.cc:260] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.702186 18244 raft_consensus.cc:385] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:44.726465 18244 raft_consensus.cc:740] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:44.727499 18244 consensus_queue.cc:260] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
W20260709 13:17:44.730361 17861 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 317953us
W20260709 13:17:44.731528 17860 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 454779us
W20260709 13:17:44.739198 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:44.744963 18037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb77b44e31244ec29dad965bc258249a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.755661 18293 raft_consensus.cc:2804] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:44.756112 18293 raft_consensus.cc:493] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.756410 18293 raft_consensus.cc:3060] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.761579 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:44.763149 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30604e2cd10b450fb3e8d52bee20f8da" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:44.707248 18200 raft_consensus.cc:2468] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:44.764688 17940 leader_election.cc:304] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:44.764966 18247 tablet_bootstrap.cc:492] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
W20260709 13:17:44.754609 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 114977us
W20260709 13:17:44.771092 18164 leader_election.cc:343] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:44.811291 17860 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 44760.7us
W20260709 13:17:44.792444 18089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 207255us
I20260709 13:17:44.827021 18254 raft_consensus.cc:2749] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:44.828748 18126 raft_consensus.cc:2468] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:44.831202 18280 raft_consensus.cc:515] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:44.831859 18244 ts_tablet_manager.cc:1434] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.166s	user 0.011s	sys 0.003s
I20260709 13:17:44.833220 18244 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:44.834115 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "843fde44d03641ce89e8d2e247c68b2d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:44.834695 17976 raft_consensus.cc:3060] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:44.835793 17943 leader_election.cc:343] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.836289 17943 leader_election.cc:304] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:44.837139 18256 raft_consensus.cc:2749] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:44.807076 18252 raft_consensus.cc:2804] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:44.804414 17861 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 63954.7us
I20260709 13:17:44.842671 18295 raft_consensus.cc:2749] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:44.819686 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:44.833323 18280 leader_election.cc:290] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:44.856292 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "843fde44d03641ce89e8d2e247c68b2d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:44.860343 18253 raft_consensus.cc:493] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.860884 18253 raft_consensus.cc:515] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:44.863270 18253 leader_election.cc:290] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:44.876871 18125 raft_consensus.cc:2468] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:44.888918 18245 ts_tablet_manager.cc:1434] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.342s	user 0.010s	sys 0.024s
I20260709 13:17:44.890369 18245 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:44.896888 18293 raft_consensus.cc:515] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.885030 18252 raft_consensus.cc:697] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:44.907714 18252 consensus_queue.cc:237] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.911044 18280 raft_consensus.cc:493] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.911568 18280 raft_consensus.cc:515] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.913671 18280 leader_election.cc:290] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:44.916611 17976 raft_consensus.cc:2468] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:44.920019 18165 leader_election.cc:304] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:44.920711 18296 raft_consensus.cc:493] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.921001 18280 raft_consensus.cc:2804] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:44.921303 18296 raft_consensus.cc:515] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:44.921780 18280 raft_consensus.cc:697] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:44.922683 18280 consensus_queue.cc:237] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:44.925626 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:44.926347 17977 raft_consensus.cc:2468] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:44.905947 18244 tablet_bootstrap.cc:654] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:44.929883 18164 leader_election.cc:343] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.934563 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247cfa8622e49e29ad0bfba76525834" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:44.935846 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:44.936652 18125 raft_consensus.cc:2393] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc8a9ec31a5745d69dae4262b271e568 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:44.936961 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:44.928419 18254 raft_consensus.cc:493] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.937435 18091 leader_election.cc:304] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:44.927853 18296 leader_election.cc:290] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:44.932229 18247 tablet_bootstrap.cc:654] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:44.937805 18254 raft_consensus.cc:515] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.940019 18166 leader_election.cc:304] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:44.940948 18297 raft_consensus.cc:3060] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.934684 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.941912 18052 raft_consensus.cc:2468] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:44.942358 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:44.941256 18296 raft_consensus.cc:2804] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:44.942941 18200 raft_consensus.cc:2468] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:44.943419 18296 raft_consensus.cc:493] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.943778 18296 raft_consensus.cc:3060] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.944419 18126 raft_consensus.cc:2468] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:44.946439 18166 leader_election.cc:304] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:44.947242 18295 raft_consensus.cc:2804] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:44.947183 18089 leader_election.cc:304] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:44.947765 18295 raft_consensus.cc:493] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.948104 18295 raft_consensus.cc:3060] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.948477 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:44.941064 18254 leader_election.cc:290] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
W20260709 13:17:44.950991 18090 leader_election.cc:343] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.953526 18254 raft_consensus.cc:2804] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:44.953904 18254 raft_consensus.cc:493] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.954232 18254 raft_consensus.cc:3060] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:44.955884 18293 leader_election.cc:290] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:44.957059 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:44.957556 18295 raft_consensus.cc:515] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:44.955662 18297 raft_consensus.cc:2749] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:44.955745 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:44.959939 18246 ts_tablet_manager.cc:1434] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.334s	user 0.026s	sys 0.028s
W20260709 13:17:44.969040 18090 leader_election.cc:343] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.970721 18246 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:44.973299 18254 raft_consensus.cc:515] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:44.978092 18017 leader_election.cc:343] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:44.974421 17871 catalog_manager.cc:5697] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:44.991221 18245 tablet_bootstrap.cc:654] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:44.979983 18293 raft_consensus.cc:493] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.990182 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:44.979769 18200 raft_consensus.cc:2468] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:44.999390 18246 tablet_bootstrap.cc:654] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:44.993125 18254 leader_election.cc:290] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:44.996798 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:45.002933 18247 tablet_bootstrap.cc:492] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:45.008503 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:45.009464 18126 raft_consensus.cc:2393] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc8a9ec31a5745d69dae4262b271e568 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:45.010919 18295 leader_election.cc:290] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:45.001919 18293 raft_consensus.cc:515] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.012234 18199 raft_consensus.cc:2393] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:45.000509 18089 leader_election.cc:304] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:45.014765 18293 leader_election.cc:290] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:45.015309 18254 raft_consensus.cc:2804] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.015782 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:45.015583 18090 leader_election.cc:343] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.019868 18090 leader_election.cc:304] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.020413 18244 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:45.020973 18244 ts_tablet_manager.cc:1403] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.188s	user 0.005s	sys 0.011s
I20260709 13:17:45.018666 18247 ts_tablet_manager.cc:1403] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.254s	user 0.015s	sys 0.005s
I20260709 13:17:45.016921 18254 raft_consensus.cc:697] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:45.024243 18293 raft_consensus.cc:2749] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:17:45.019791 18091 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 86236.3us
I20260709 13:17:45.025679 18296 raft_consensus.cc:515] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.033564 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247cfa8622e49e29ad0bfba76525834" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:45.028764 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:45.035859 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:45.037357 18090 leader_election.cc:343] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:45.037932 18089 leader_election.cc:343] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.038247 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:45.038359 18089 leader_election.cc:304] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.039775 18302 raft_consensus.cc:2749] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:45.040037 18015 leader_election.cc:343] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:45.040064 18164 leader_election.cc:343] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:45.041247 18164 leader_election.cc:343] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.041559 18164 leader_election.cc:304] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:45.037353 17872 catalog_manager.cc:5697] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.042116 18244 raft_consensus.cc:359] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.042882 18244 raft_consensus.cc:385] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.042490 18280 raft_consensus.cc:2749] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.040495 18015 leader_election.cc:304] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.045656 18253 raft_consensus.cc:2749] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.038383 18247 raft_consensus.cc:359] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.046911 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:45.047550 18052 raft_consensus.cc:3060] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.047600 18247 raft_consensus.cc:385] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.048224 18247 raft_consensus.cc:740] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.048429 18254 consensus_queue.cc:237] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.049000 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:45.049705 17977 raft_consensus.cc:3060] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.049840 18247 consensus_queue.cc:260] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.054931 18052 raft_consensus.cc:2468] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:45.043172 18244 raft_consensus.cc:740] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.056069 18090 leader_election.cc:304] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:45.056695 18296 leader_election.cc:290] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:45.057039 18293 raft_consensus.cc:2804] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.057600 18293 raft_consensus.cc:697] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:45.058509 18293 consensus_queue.cc:237] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.056005 18244 consensus_queue.cc:260] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.061003 18247 ts_tablet_manager.cc:1434] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.002s
I20260709 13:17:45.061295 18244 ts_tablet_manager.cc:1434] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.001s
I20260709 13:17:45.073210 18247 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:45.073184 18244 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:45.071944 17977 raft_consensus.cc:2468] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:45.091114 18245 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:45.091750 18245 ts_tablet_manager.cc:1403] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.202s	user 0.023s	sys 0.001s
I20260709 13:17:45.095098 18245 raft_consensus.cc:359] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.096278 18245 raft_consensus.cc:385] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.096674 18245 raft_consensus.cc:740] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.097801 18244 tablet_bootstrap.cc:654] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.097724 18245 consensus_queue.cc:260] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.100334 18245 ts_tablet_manager.cc:1434] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:45.102159 18246 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:45.103549 18247 tablet_bootstrap.cc:654] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.103886 18246 ts_tablet_manager.cc:1403] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.133s	user 0.018s	sys 0.010s
I20260709 13:17:45.110193 18246 raft_consensus.cc:359] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.112535 18245 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:45.115263 18244 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:45.116026 18244 ts_tablet_manager.cc:1403] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.000s
I20260709 13:17:45.112094 18246 raft_consensus.cc:385] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.117061 18246 raft_consensus.cc:740] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.118338 18246 consensus_queue.cc:260] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.119626 18244 raft_consensus.cc:359] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.121217 18244 raft_consensus.cc:385] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.121237 18246 ts_tablet_manager.cc:1434] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.005s
I20260709 13:17:45.122025 18244 raft_consensus.cc:740] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.123034 18246 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:45.124514 18245 tablet_bootstrap.cc:654] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.124141 18244 consensus_queue.cc:260] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.128041 18244 ts_tablet_manager.cc:1434] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:17:45.131197 18244 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:45.139005 18247 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:45.139650 18247 ts_tablet_manager.cc:1403] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.004s
I20260709 13:17:45.142935 18253 raft_consensus.cc:493] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.143469 18253 raft_consensus.cc:515] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.145774 18253 leader_election.cc:290] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:45.147348 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:45.147786 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97db5bc48fc437fac40454dc3e1d871" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:45.148252 18199 raft_consensus.cc:2393] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:45.149746 18018 leader_election.cc:304] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.150218 18247 raft_consensus.cc:359] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.150986 18247 raft_consensus.cc:385] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.151252 18247 raft_consensus.cc:740] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.152490 18253 raft_consensus.cc:3060] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.154937 18245 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:45.155604 18245 ts_tablet_manager.cc:1403] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.000s
I20260709 13:17:45.157101 18246 tablet_bootstrap.cc:654] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.157545 18247 consensus_queue.cc:260] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.159377 18245 raft_consensus.cc:359] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.160395 18245 raft_consensus.cc:385] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.160689 18245 raft_consensus.cc:740] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.162693 18245 consensus_queue.cc:260] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.165369 18245 ts_tablet_manager.cc:1434] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:45.166723 18245 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:45.168246 18253 raft_consensus.cc:2749] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.155014 18244 tablet_bootstrap.cc:654] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.179692 18246 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:45.180248 18246 ts_tablet_manager.cc:1403] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.008s
I20260709 13:17:45.182217 17871 catalog_manager.cc:5697] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.183283 18246 raft_consensus.cc:359] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.195309 18246 raft_consensus.cc:385] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.195624 18246 raft_consensus.cc:740] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.196516 18246 consensus_queue.cc:260] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.188807 18245 tablet_bootstrap.cc:654] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.204121 18247 ts_tablet_manager.cc:1434] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.023s
I20260709 13:17:45.205602 18247 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:45.206073 18246 ts_tablet_manager.cc:1434] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:17:45.196918 17871 catalog_manager.cc:5697] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.223050 18247 tablet_bootstrap.cc:654] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.227722 18244 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:45.228261 18244 ts_tablet_manager.cc:1403] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.008s
I20260709 13:17:45.231343 18244 raft_consensus.cc:359] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.232172 18244 raft_consensus.cc:385] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.232451 18244 raft_consensus.cc:740] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.233596 18244 consensus_queue.cc:260] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.235519 18246 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:45.239768 18244 ts_tablet_manager.cc:1434] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:17:45.241619 18244 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:45.251610 18252 raft_consensus.cc:493] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.252278 18252 raft_consensus.cc:515] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.254558 18246 tablet_bootstrap.cc:654] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.256234 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:45.258844 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:45.262457 17941 leader_election.cc:343] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:45.263964 17943 leader_election.cc:343] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.264482 17943 leader_election.cc:304] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:45.265595 18298 raft_consensus.cc:2749] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.267063 18298 raft_consensus.cc:493] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.267840 18245 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:45.267602 18298 raft_consensus.cc:515] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.268402 18245 ts_tablet_manager.cc:1403] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.102s	user 0.024s	sys 0.036s
I20260709 13:17:45.270545 18298 leader_election.cc:290] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:45.270725 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:45.271426 18199 raft_consensus.cc:2468] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:45.271826 18245 raft_consensus.cc:359] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.272537 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:45.272732 17940 leader_election.cc:304] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:45.272557 18245 raft_consensus.cc:385] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.273573 18245 raft_consensus.cc:740] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.273597 18298 raft_consensus.cc:2804] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.274176 18298 raft_consensus.cc:493] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:45.274298 17941 leader_election.cc:343] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.274523 18245 consensus_queue.cc:260] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.274554 18298 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.254504 18252 leader_election.cc:290] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
W20260709 13:17:45.281298 17906 debug-util.cc:398] Leaking SignalData structure 0x7b0800195240 after lost signal to thread 10260
I20260709 13:17:45.282514 18247 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
W20260709 13:17:45.282637 17906 debug-util.cc:398] Leaking SignalData structure 0x7b0800219ae0 after lost signal to thread 17909
W20260709 13:17:45.284168 17906 debug-util.cc:398] Leaking SignalData structure 0x7b080016e500 after lost signal to thread 18001
I20260709 13:17:45.285074 18245 ts_tablet_manager.cc:1434] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
W20260709 13:17:45.285413 17906 debug-util.cc:398] Leaking SignalData structure 0x7b080012f9c0 after lost signal to thread 18076
W20260709 13:17:45.285733 17906 debug-util.cc:398] Leaking SignalData structure 0x7b080012fa00 after lost signal to thread 18077
W20260709 13:17:45.286901 17906 debug-util.cc:398] Leaking SignalData structure 0x7b080032ac80 after lost signal to thread 18151
I20260709 13:17:45.287747 18246 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:45.288252 18246 ts_tablet_manager.cc:1403] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.004s
W20260709 13:17:45.288581 17906 debug-util.cc:398] Leaking SignalData structure 0x7b08002278a0 after lost signal to thread 18224
I20260709 13:17:45.290469 18245 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:45.289264 18247 ts_tablet_manager.cc:1403] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.005s
I20260709 13:17:45.294322 18247 raft_consensus.cc:359] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.297516 18245 tablet_bootstrap.cc:654] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.283052 18298 raft_consensus.cc:515] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.299068 18246 raft_consensus.cc:359] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.299629 18244 tablet_bootstrap.cc:654] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.304337 18052 raft_consensus.cc:3060] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.305107 18246 raft_consensus.cc:385] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.305451 18246 raft_consensus.cc:740] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.306492 18298 leader_election.cc:290] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:45.306181 18246 consensus_queue.cc:260] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.308404 18246 ts_tablet_manager.cc:1434] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:17:45.308668 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:45.310321 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
W20260709 13:17:45.310475 17941 leader_election.cc:343] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.310899 18199 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.318234 18280 consensus_queue.cc:1048] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:45.319448 18246 tablet_bootstrap.cc:492] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:45.325578 18246 tablet_bootstrap.cc:654] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.304944 18247 raft_consensus.cc:385] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.329118 18247 raft_consensus.cc:740] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.330907 18247 consensus_queue.cc:260] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.318506 18199 raft_consensus.cc:2468] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:45.335747 18247 ts_tablet_manager.cc:1434] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 13:17:45.336912 18247 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:45.337181 17940 leader_election.cc:304] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:45.338110 18298 raft_consensus.cc:2804] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.338635 18298 raft_consensus.cc:697] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:45.339649 18298 consensus_queue.cc:237] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.358739 18244 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:45.359331 18244 ts_tablet_manager.cc:1403] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.118s	user 0.009s	sys 0.004s
I20260709 13:17:45.361550 18280 consensus_queue.cc:1048] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.367568 18252 consensus_queue.cc:1048] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.376128 17871 catalog_manager.cc:5697] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.380752 18246 tablet_bootstrap.cc:492] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:45.382726 18246 ts_tablet_manager.cc:1403] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.004s
I20260709 13:17:45.384112 18293 raft_consensus.cc:493] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.384667 18293 raft_consensus.cc:515] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.386905 18293 leader_election.cc:290] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.388672 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad41548503545059d74bd2564c17b52" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:45.389338 18247 tablet_bootstrap.cc:654] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.389171 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad41548503545059d74bd2564c17b52" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:45.390447 18089 leader_election.cc:343] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:45.391112 18091 leader_election.cc:343] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.391671 18091 leader_election.cc:304] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.393014 18293 raft_consensus.cc:2749] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.384752 18244 raft_consensus.cc:359] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.396170 18244 raft_consensus.cc:385] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.396584 18244 raft_consensus.cc:740] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.397451 18244 consensus_queue.cc:260] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.396087 18245 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:45.405851 18245 ts_tablet_manager.cc:1403] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.116s	user 0.026s	sys 0.043s
I20260709 13:17:45.387117 18246 raft_consensus.cc:359] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.408958 18246 raft_consensus.cc:385] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.409297 18246 raft_consensus.cc:740] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.409535 18244 ts_tablet_manager.cc:1434] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.003s
I20260709 13:17:45.410640 18244 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:45.410224 18246 consensus_queue.cc:260] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.417037 18246 ts_tablet_manager.cc:1434] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.002s
I20260709 13:17:45.417315 18244 tablet_bootstrap.cc:654] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.418291 18246 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:45.432224 18246 tablet_bootstrap.cc:654] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.424237 18245 raft_consensus.cc:359] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.435585 18245 raft_consensus.cc:385] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.435865 18245 raft_consensus.cc:740] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.436708 18245 consensus_queue.cc:260] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.439141 18245 ts_tablet_manager.cc:1434] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.033s	user 0.002s	sys 0.007s
I20260709 13:17:45.440276 18245 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:45.443648 18252 consensus_queue.cc:1048] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.444043 18244 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:45.444599 18244 ts_tablet_manager.cc:1403] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 13:17:45.440263 18247 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:45.445497 18247 ts_tablet_manager.cc:1403] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.109s	user 0.014s	sys 0.016s
I20260709 13:17:45.447243 18245 tablet_bootstrap.cc:654] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.462809 18244 raft_consensus.cc:359] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.463734 18244 raft_consensus.cc:385] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.464006 18244 raft_consensus.cc:740] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.476058 18247 raft_consensus.cc:359] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.476619 18293 consensus_queue.cc:1048] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.477277 18247 raft_consensus.cc:385] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.477734 18247 raft_consensus.cc:740] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.478718 18247 consensus_queue.cc:260] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.480625 18244 consensus_queue.cc:260] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.484081 18244 ts_tablet_manager.cc:1434] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:17:45.484243 18247 ts_tablet_manager.cc:1434] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:17:45.490549 18247 tablet_bootstrap.cc:492] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:45.496824 18244 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:45.510293 18296 consensus_queue.cc:1048] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.517503 18247 tablet_bootstrap.cc:654] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.517644 18252 raft_consensus.cc:493] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.518172 18252 raft_consensus.cc:515] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.519783 18246 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:45.520262 18246 ts_tablet_manager.cc:1403] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.102s	user 0.014s	sys 0.016s
I20260709 13:17:45.520925 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:45.521581 18200 raft_consensus.cc:2468] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:45.526448 17940 leader_election.cc:304] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:45.532073 18252 leader_election.cc:290] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:45.532708 18252 raft_consensus.cc:2804] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.533039 18252 raft_consensus.cc:493] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.533255 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:45.537292 18246 raft_consensus.cc:359] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.538118 18246 raft_consensus.cc:385] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.538406 18246 raft_consensus.cc:740] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.533360 18252 raft_consensus.cc:3060] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.548172 18319 raft_consensus.cc:493] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:45.545317 17941 leader_election.cc:343] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.548799 18319 raft_consensus.cc:515] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.550904 18319 leader_election.cc:290] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:45.552464 18246 consensus_queue.cc:260] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.563006 18280 raft_consensus.cc:493] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.563580 18280 raft_consensus.cc:515] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.563983 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cca6423ee4465cb73609e4e335e66e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:45.564709 18125 raft_consensus.cc:2468] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:45.566030 18018 leader_election.cc:304] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:45.567414 18319 raft_consensus.cc:2804] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.567765 18319 raft_consensus.cc:493] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.567873 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247cfa8622e49e29ad0bfba76525834" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:45.568441 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247cfa8622e49e29ad0bfba76525834" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:45.565924 18280 leader_election.cc:290] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.569051 18052 raft_consensus.cc:2468] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
W20260709 13:17:45.569659 18165 leader_election.cc:343] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.570274 18319 raft_consensus.cc:3060] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.570537 18164 leader_election.cc:304] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:45.571782 18280 raft_consensus.cc:2804] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.572209 18280 raft_consensus.cc:493] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.572649 18280 raft_consensus.cc:3060] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.578022 18319 raft_consensus.cc:515] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.563509 18298 raft_consensus.cc:493] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.579258 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cca6423ee4465cb73609e4e335e66e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:45.579445 18298 raft_consensus.cc:515] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:17:45.580897 18017 leader_election.cc:343] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.583488 18244 tablet_bootstrap.cc:654] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.587217 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:45.578886 18252 raft_consensus.cc:515] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.592334 18246 ts_tablet_manager.cc:1434] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.003s
W20260709 13:17:45.592310 17943 leader_election.cc:343] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.596331 18298 leader_election.cc:290] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:45.597203 18319 leader_election.cc:290] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:45.597937 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cca6423ee4465cb73609e4e335e66e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:45.599462 18245 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:45.600050 18245 ts_tablet_manager.cc:1403] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.160s	user 0.015s	sys 0.003s
I20260709 13:17:45.600747 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:45.602300 17940 leader_election.cc:343] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.602735 17940 leader_election.cc:304] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
W20260709 13:17:45.599382 18017 leader_election.cc:343] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.603245 18280 raft_consensus.cc:515] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.612067 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247cfa8622e49e29ad0bfba76525834" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:45.612622 18052 raft_consensus.cc:3060] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.613603 18254 consensus_queue.cc:1048] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.602752 18245 raft_consensus.cc:359] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.603647 18345 raft_consensus.cc:2749] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.604743 18246 tablet_bootstrap.cc:492] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:45.606698 18293 raft_consensus.cc:493] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.609571 18296 raft_consensus.cc:493] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.616044 18245 raft_consensus.cc:385] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.618221 18245 raft_consensus.cc:740] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.627192 18052 raft_consensus.cc:2468] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:45.618849 18296 raft_consensus.cc:515] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.633323 18296 leader_election.cc:290] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.621536 18252 leader_election.cc:290] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:45.622659 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:45.622181 18280 leader_election.cc:290] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.625521 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:45.635075 18244 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:45.618547 18293 raft_consensus.cc:515] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.635128 18200 raft_consensus.cc:3060] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.635787 18244 ts_tablet_manager.cc:1403] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.140s	user 0.020s	sys 0.004s
I20260709 13:17:45.626899 18254 consensus_queue.cc:1048] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.638758 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:45.639073 17941 leader_election.cc:343] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.639204 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30604e2cd10b450fb3e8d52bee20f8da" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:45.639503 18048 raft_consensus.cc:2468] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:45.639869 18198 raft_consensus.cc:2468] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:45.637202 18293 leader_election.cc:290] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.640311 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:45.640735 18090 leader_election.cc:304] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:45.641093 18089 leader_election.cc:304] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:45.641615 18254 raft_consensus.cc:2804] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.641640 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30604e2cd10b450fb3e8d52bee20f8da" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:45.642418 18254 raft_consensus.cc:493] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.643020 18254 raft_consensus.cc:3060] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:45.644992 18091 leader_election.cc:343] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.642916 17975 raft_consensus.cc:2468] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:45.647367 18296 raft_consensus.cc:2804] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.647805 18296 raft_consensus.cc:493] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.648160 18296 raft_consensus.cc:3060] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.627988 18245 consensus_queue.cc:260] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.653079 18164 leader_election.cc:304] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:45.630697 18246 tablet_bootstrap.cc:654] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.639235 18244 raft_consensus.cc:359] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.633447 18247 tablet_bootstrap.cc:492] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:45.654001 18244 raft_consensus.cc:385] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.654245 18247 ts_tablet_manager.cc:1403] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.164s	user 0.019s	sys 0.001s
I20260709 13:17:45.654286 18244 raft_consensus.cc:740] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.630245 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6247cfa8622e49e29ad0bfba76525834" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:45.656718 18244 consensus_queue.cc:260] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.658120 18280 raft_consensus.cc:2804] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.658675 18280 raft_consensus.cc:697] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:45.659320 18244 ts_tablet_manager.cc:1434] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.002s
I20260709 13:17:45.659569 18280 consensus_queue.cc:237] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.660238 18247 raft_consensus.cc:359] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.660825 18244 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:17:45.661839 18165 leader_election.cc:343] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.662803 18247 raft_consensus.cc:385] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.663097 18247 raft_consensus.cc:740] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.663566 18295 raft_consensus.cc:493] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.664058 18295 raft_consensus.cc:515] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.663827 18247 consensus_queue.cc:260] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.664678 18245 ts_tablet_manager.cc:1434] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 13:17:45.665784 18245 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:45.666388 18295 leader_election.cc:290] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:45.666693 18247 ts_tablet_manager.cc:1434] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:17:45.667708 18247 tablet_bootstrap.cc:492] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:45.668152 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:45.668780 18125 raft_consensus.cc:2468] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:45.657545 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cca6423ee4465cb73609e4e335e66e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:45.669870 18126 raft_consensus.cc:3060] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.670473 18296 raft_consensus.cc:515] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.674216 18166 leader_election.cc:304] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:45.674638 18244 tablet_bootstrap.cc:654] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.676039 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:45.681629 18295 raft_consensus.cc:2804] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.686702 18295 raft_consensus.cc:493] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.687045 18295 raft_consensus.cc:3060] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:45.691955 18164 leader_election.cc:343] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.701689 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30604e2cd10b450fb3e8d52bee20f8da" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:45.702374 18198 raft_consensus.cc:3060] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.708985 18200 raft_consensus.cc:2468] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:45.709944 18254 raft_consensus.cc:515] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.712292 17940 leader_election.cc:304] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:45.713778 18346 raft_consensus.cc:2804] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.713980 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
W20260709 13:17:45.715878 18091 leader_election.cc:343] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.714354 18346 raft_consensus.cc:697] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:45.716264 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:45.716907 18052 raft_consensus.cc:3060] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.717082 18346 consensus_queue.cc:237] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.718510 18295 raft_consensus.cc:515] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.720844 18295 leader_election.cc:290] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:45.721752 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
W20260709 13:17:45.723649 18164 leader_election.cc:343] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.724401 18052 raft_consensus.cc:2468] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:45.725622 18090 leader_election.cc:304] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:45.724643 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:45.726483 18125 raft_consensus.cc:3060] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.727947 18296 leader_election.cc:290] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.729404 18245 tablet_bootstrap.cc:654] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.726542 18293 raft_consensus.cc:2804] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.732858 18293 raft_consensus.cc:697] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:45.735555 18293 consensus_queue.cc:237] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.740480 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30604e2cd10b450fb3e8d52bee20f8da" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:45.712730 18254 leader_election.cc:290] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.741181 17977 raft_consensus.cc:3060] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.754920 18246 tablet_bootstrap.cc:492] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:45.756393 18246 ts_tablet_manager.cc:1403] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.152s	user 0.011s	sys 0.009s
I20260709 13:17:45.757035 18252 consensus_queue.cc:1048] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:45.731884 18126 raft_consensus.cc:2468] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
W20260709 13:17:45.764933 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:45.765383 18018 leader_election.cc:304] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:45.756817 18198 raft_consensus.cc:2468] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:45.769147 18089 leader_election.cc:304] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:45.756794 17873 catalog_manager.cc:5697] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.767933 18319 raft_consensus.cc:2804] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.782819 18246 raft_consensus.cc:359] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.788450 18246 raft_consensus.cc:385] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.788733 18246 raft_consensus.cc:740] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.787762 18125 raft_consensus.cc:2468] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:45.794705 18166 leader_election.cc:304] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:45.796051 18280 raft_consensus.cc:2804] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.796635 18280 raft_consensus.cc:697] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:45.797505 18280 consensus_queue.cc:237] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.801648 17872 catalog_manager.cc:5697] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.803442 18245 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:45.804055 18245 ts_tablet_manager.cc:1403] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.139s	user 0.015s	sys 0.025s
I20260709 13:17:45.809298 18244 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:45.811897 18244 ts_tablet_manager.cc:1403] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.151s	user 0.018s	sys 0.041s
I20260709 13:17:45.811834 18319 raft_consensus.cc:697] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:45.830247 18247 tablet_bootstrap.cc:654] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.782001 17977 raft_consensus.cc:2468] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:45.835085 18244 raft_consensus.cc:359] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.835593 18355 raft_consensus.cc:2804] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.851869 18244 raft_consensus.cc:385] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.852331 18244 raft_consensus.cc:740] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.836802 18319 consensus_queue.cc:237] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.837285 18051 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.838652 17873 catalog_manager.cc:5697] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.841562 18295 raft_consensus.cc:493] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.853402 18295 raft_consensus.cc:515] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.852367 18355 raft_consensus.cc:697] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:45.855177 18244 consensus_queue.cc:260] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.858275 18244 ts_tablet_manager.cc:1434] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.046s	user 0.011s	sys 0.007s
I20260709 13:17:45.814167 18246 consensus_queue.cc:260] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.855085 18355 consensus_queue.cc:237] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.806865 18245 raft_consensus.cc:359] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.855993 18295 leader_election.cc:290] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.855993 18296 raft_consensus.cc:493] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.860576 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad41548503545059d74bd2564c17b52" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:45.870949 18245 raft_consensus.cc:385] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.871368 18245 raft_consensus.cc:740] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.871443 18296 raft_consensus.cc:515] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.872468 18245 consensus_queue.cc:260] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:45.873898 18296 leader_election.cc:290] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.860579 18246 ts_tablet_manager.cc:1434] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.104s	user 0.005s	sys 0.002s
I20260709 13:17:45.856211 18346 raft_consensus.cc:493] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.874825 18346 raft_consensus.cc:515] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.875322 18246 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:45.875844 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:45.877274 18089 leader_election.cc:343] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.878978 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:45.860844 18361 raft_consensus.cc:493] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.871908 18125 raft_consensus.cc:2468] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:45.879386 18346 leader_election.cc:290] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:45.883317 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:45.870990 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad41548503545059d74bd2564c17b52" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:45.885108 17941 leader_election.cc:343] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.886039 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:45.886222 18165 leader_election.cc:343] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:45.887758 17940 leader_election.cc:343] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.888248 17940 leader_election.cc:304] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.889132 18252 raft_consensus.cc:2749] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.891938 17975 raft_consensus.cc:2468] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:45.892714 18166 leader_election.cc:304] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:45.894158 18295 raft_consensus.cc:2804] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.894657 18295 raft_consensus.cc:493] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.895213 18295 raft_consensus.cc:3060] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.883421 18245 ts_tablet_manager.cc:1434] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.007s
I20260709 13:17:45.897893 18245 tablet_bootstrap.cc:492] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:45.899767 18091 leader_election.cc:304] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.900295 18252 consensus_queue.cc:1048] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:45.902792 18293 raft_consensus.cc:2804] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.903192 18293 raft_consensus.cc:493] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.903519 18293 raft_consensus.cc:3060] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.904542 18244 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:45.892926 18361 raft_consensus.cc:515] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.914865 18361 leader_election.cc:290] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:45.915725 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f30e4471448411bab26f97e6d34f33c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:45.915989 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f30e4471448411bab26f97e6d34f33c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:45.916672 18199 raft_consensus.cc:2468] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
W20260709 13:17:45.917320 18018 leader_election.cc:343] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.917915 18015 leader_election.cc:304] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:45.918615 18245 tablet_bootstrap.cc:654] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.919138 18361 raft_consensus.cc:2804] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.919601 18361 raft_consensus.cc:493] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.920017 18361 raft_consensus.cc:3060] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.926952 17871 catalog_manager.cc:5697] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.927625 18293 raft_consensus.cc:515] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.931197 18246 tablet_bootstrap.cc:654] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.927501 18244 tablet_bootstrap.cc:654] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.956692 18293 leader_election.cc:290] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:45.957219 18247 tablet_bootstrap.cc:492] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:45.958021 18247 ts_tablet_manager.cc:1403] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.290s	user 0.016s	sys 0.006s
I20260709 13:17:45.959230 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:45.957368 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:45.960083 18295 raft_consensus.cc:515] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.961225 18247 raft_consensus.cc:359] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:45.961735 18089 leader_election.cc:343] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.959924 17975 raft_consensus.cc:3060] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.962097 18247 raft_consensus.cc:385] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.962517 18247 raft_consensus.cc:740] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.963366 18247 consensus_queue.cc:260] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:45.963804 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad41548503545059d74bd2564c17b52" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:45.964430 18125 raft_consensus.cc:3060] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.966073 18247 ts_tablet_manager.cc:1434] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:45.967231 18247 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:45.973333 18247 tablet_bootstrap.cc:654] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.973556 18125 raft_consensus.cc:2468] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:45.974756 18166 leader_election.cc:304] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:45.975553 18297 raft_consensus.cc:2804] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.976032 18297 raft_consensus.cc:697] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:45.976919 18297 consensus_queue.cc:237] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.961360 18358 raft_consensus.cc:493] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:45.985594 17975 raft_consensus.cc:2468] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:45.986845 17872 catalog_manager.cc:5697] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.989444 18091 leader_election.cc:304] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:45.990337 18293 raft_consensus.cc:2804] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.990828 18293 raft_consensus.cc:697] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:45.982154 18358 raft_consensus.cc:515] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:45.991928 18293 consensus_queue.cc:237] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:45.998054 18246 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:45.998708 18246 ts_tablet_manager.cc:1403] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.124s	user 0.032s	sys 0.010s
I20260709 13:17:46.001813 18246 raft_consensus.cc:359] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.002574 18246 raft_consensus.cc:385] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.002847 18246 raft_consensus.cc:740] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.003607 18246 consensus_queue.cc:260] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.011518 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:46.014621 18358 leader_election.cc:290] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:46.015476 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:46.016443 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ad41548503545059d74bd2564c17b52" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:46.017192 18017 leader_election.cc:304] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
W20260709 13:17:46.018263 18165 leader_election.cc:343] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.019418 18246 ts_tablet_manager.cc:1434] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:17:46.020869 18246 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:46.021150 18295 leader_election.cc:290] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:46.028218 18358 raft_consensus.cc:2749] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:46.033958 18361 raft_consensus.cc:515] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.034641 18246 tablet_bootstrap.cc:654] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.036895 18247 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:46.037413 18247 ts_tablet_manager.cc:1403] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.070s	user 0.007s	sys 0.007s
I20260709 13:17:46.044823 18247 raft_consensus.cc:359] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.045506 18247 raft_consensus.cc:385] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.045759 18247 raft_consensus.cc:740] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.046008 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f30e4471448411bab26f97e6d34f33c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:46.046334 18247 consensus_queue.cc:260] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
W20260709 13:17:46.059062 18018 leader_election.cc:343] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.059900 17871 catalog_manager.cc:5697] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.066802 18247 ts_tablet_manager.cc:1434] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:17:46.068142 17872 catalog_manager.cc:5697] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.075280 18344 raft_consensus.cc:493] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.075871 18344 raft_consensus.cc:515] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.076397 18361 leader_election.cc:290] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:46.078097 18344 leader_election.cc:290] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.078326 18247 tablet_bootstrap.cc:492] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:46.080235 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f30e4471448411bab26f97e6d34f33c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:46.080845 18200 raft_consensus.cc:3060] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.083870 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.084535 18125 raft_consensus.cc:2468] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:46.085858 17943 leader_election.cc:304] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:46.086683 18346 raft_consensus.cc:2804] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:46.087023 18346 raft_consensus.cc:493] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.087373 18346 raft_consensus.cc:3060] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.093524 18245 tablet_bootstrap.cc:492] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:46.094056 18245 ts_tablet_manager.cc:1403] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.196s	user 0.004s	sys 0.014s
W20260709 13:17:46.096182 18165 consensus_peers.cc:597] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:46.097084 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:46.098618 17941 leader_election.cc:343] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.099440 18245 raft_consensus.cc:359] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.100279 18245 raft_consensus.cc:385] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.100579 18245 raft_consensus.cc:740] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.101392 18245 consensus_queue.cc:260] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.102999 18244 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:46.103513 18244 ts_tablet_manager.cc:1403] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.199s	user 0.008s	sys 0.006s
I20260709 13:17:46.105142 18247 tablet_bootstrap.cc:654] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.105662 18245 ts_tablet_manager.cc:1434] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:46.106786 18245 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:46.111647 18244 raft_consensus.cc:359] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.112337 18244 raft_consensus.cc:385] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.112600 18244 raft_consensus.cc:740] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.113413 18244 consensus_queue.cc:260] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.115490 18244 ts_tablet_manager.cc:1434] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:17:46.116778 18200 raft_consensus.cc:2468] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:46.117473 18246 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:46.117990 18246 ts_tablet_manager.cc:1403] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.097s	user 0.021s	sys 0.034s
I20260709 13:17:46.118372 18015 leader_election.cc:304] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:46.119234 18361 raft_consensus.cc:2804] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:46.119814 18361 raft_consensus.cc:697] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:46.120545 18346 raft_consensus.cc:515] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.121431 18244 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:46.120671 18361 consensus_queue.cc:237] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.121836 18246 raft_consensus.cc:359] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.124027 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:46.125402 18346 leader_election.cc:290] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
W20260709 13:17:46.125798 17941 leader_election.cc:343] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.126438 18246 raft_consensus.cc:385] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.126784 18246 raft_consensus.cc:740] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.126645 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:46.127559 18125 raft_consensus.cc:3060] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.127866 18246 consensus_queue.cc:260] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.130504 18246 ts_tablet_manager.cc:1434] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:17:46.131650 18246 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:46.134825 17873 catalog_manager.cc:5697] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.137459 18293 consensus_queue.cc:1048] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.138576 18125 raft_consensus.cc:2468] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:46.139719 17943 leader_election.cc:304] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:46.140602 17977 raft_consensus.cc:3060] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.140987 18346 raft_consensus.cc:2804] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:46.141721 18346 raft_consensus.cc:697] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:46.142572 18346 consensus_queue.cc:237] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.159327 18244 tablet_bootstrap.cc:654] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.168000 18358 raft_consensus.cc:493] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.168723 18358 raft_consensus.cc:515] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.170959 18355 consensus_queue.cc:1048] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.172684 18247 tablet_bootstrap.cc:492] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:46.173254 18247 ts_tablet_manager.cc:1403] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.095s	user 0.031s	sys 0.000s
I20260709 13:17:46.173921 18358 leader_election.cc:290] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.175830 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:46.176641 17975 raft_consensus.cc:2468] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:46.179085 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.181037 18017 leader_election.cc:304] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
W20260709 13:17:46.182844 18018 leader_election.cc:343] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.176235 18247 raft_consensus.cc:359] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.173350 18344 raft_consensus.cc:493] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.184640 18344 raft_consensus.cc:515] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.184161 18247 raft_consensus.cc:385] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.185456 18247 raft_consensus.cc:740] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.186319 18247 consensus_queue.cc:260] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.187326 18360 raft_consensus.cc:2804] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:46.187726 18360 raft_consensus.cc:493] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.188102 18360 raft_consensus.cc:3060] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.188448 18247 ts_tablet_manager.cc:1434] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:17:46.189543 18247 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:46.193281 18319 raft_consensus.cc:493] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.193858 18319 raft_consensus.cc:515] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.196127 18319 leader_election.cc:290] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:46.197080 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:46.200546 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:46.202132 17872 catalog_manager.cc:5697] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.204417 18296 raft_consensus.cc:493] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.205152 18296 raft_consensus.cc:515] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.205516 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7241c7eca1b4c53a4faea0bcffee354" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.205554 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.204532 18360 raft_consensus.cc:515] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.206785 18125 raft_consensus.cc:2393] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:46.205278 18344 leader_election.cc:290] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.207970 18296 leader_election.cc:290] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:46.208577 17943 leader_election.cc:304] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:46.209822 18018 leader_election.cc:304] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:46.210451 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f30e4471448411bab26f97e6d34f33c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:46.211251 18200 raft_consensus.cc:2393] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:46.212392 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:46.213649 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f30e4471448411bab26f97e6d34f33c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:46.214411 18358 raft_consensus.cc:3060] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.215464 18090 leader_election.cc:304] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:46.213932 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:46.216539 17975 raft_consensus.cc:3060] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:46.216977 18018 leader_election.cc:343] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.217764 18355 raft_consensus.cc:3060] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.184108 18246 tablet_bootstrap.cc:654] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.220292 18247 tablet_bootstrap.cc:654] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.210003 18360 leader_election.cc:290] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.224457 18245 tablet_bootstrap.cc:654] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.240645 17975 raft_consensus.cc:2468] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
W20260709 13:17:46.241890 17941 consensus_peers.cc:597] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:46.242197 18017 leader_election.cc:304] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:46.243299 18360 raft_consensus.cc:2804] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:46.246357 18252 raft_consensus.cc:2749] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:46.247968 18360 raft_consensus.cc:697] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:46.248533 18297 consensus_queue.cc:1048] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.248940 18360 consensus_queue.cc:237] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.243741 18355 raft_consensus.cc:2749] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.255808 18246 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:46.255280 18358 raft_consensus.cc:2749] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.256593 18246 ts_tablet_manager.cc:1403] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.125s	user 0.011s	sys 0.002s
I20260709 13:17:46.269804 18346 consensus_queue.cc:1048] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.283274 18297 consensus_queue.cc:1048] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.282935 17873 catalog_manager.cc:5697] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.295619 18244 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:46.296103 18244 ts_tablet_manager.cc:1403] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.175s	user 0.017s	sys 0.024s
I20260709 13:17:46.299484 18358 raft_consensus.cc:493] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.300045 18358 raft_consensus.cc:515] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.301465 18247 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:46.302006 18247 ts_tablet_manager.cc:1403] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.113s	user 0.008s	sys 0.010s
I20260709 13:17:46.302229 18358 leader_election.cc:290] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:46.304065 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:46.305087 18246 raft_consensus.cc:359] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:17:46.305569 18015 leader_election.cc:343] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.306798 18246 raft_consensus.cc:385] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.307065 18246 raft_consensus.cc:740] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.309208 18297 raft_consensus.cc:493] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.310212 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.309760 18297 raft_consensus.cc:515] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.321589 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:46.323639 18018 leader_election.cc:343] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.324335 18018 leader_election.cc:304] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:46.324412 18247 raft_consensus.cc:359] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.325325 18247 raft_consensus.cc:385] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.325644 18247 raft_consensus.cc:740] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
W20260709 13:17:46.326121 18164 leader_election.cc:343] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.326417 18247 consensus_queue.cc:260] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.327527 18358 raft_consensus.cc:2749] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.327957 18244 raft_consensus.cc:359] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.328727 18244 raft_consensus.cc:385] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.328866 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:46.330421 18246 consensus_queue.cc:260] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:17:46.330861 18165 leader_election.cc:343] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.331489 18165 leader_election.cc:304] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:46.332408 18295 raft_consensus.cc:2749] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.331646 18297 leader_election.cc:290] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:46.335176 18244 raft_consensus.cc:740] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.336498 18244 consensus_queue.cc:260] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.345091 18247 ts_tablet_manager.cc:1434] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:17:46.341327 18346 raft_consensus.cc:493] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.347533 18346 raft_consensus.cc:515] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.350461 18346 leader_election.cc:290] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.351687 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cca6423ee4465cb73609e4e335e66e" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.352496 18125 raft_consensus.cc:2393] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:46.356338 18280 raft_consensus.cc:493] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.356781 18280 raft_consensus.cc:515] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.358734 18280 leader_election.cc:290] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:46.359913 18244 ts_tablet_manager.cc:1434] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.001s
I20260709 13:17:46.361028 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:46.361634 17975 raft_consensus.cc:2468] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:46.362756 18165 leader_election.cc:304] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:46.363596 18355 consensus_queue.cc:1048] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.363972 18280 raft_consensus.cc:2804] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:46.361641 18246 ts_tablet_manager.cc:1434] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.105s	user 0.006s	sys 0.001s
I20260709 13:17:46.367372 18247 tablet_bootstrap.cc:492] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:46.368436 18246 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:46.361050 18244 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:46.368438 18280 raft_consensus.cc:493] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.369763 18280 raft_consensus.cc:3060] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.371258 18245 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:46.371773 18245 ts_tablet_manager.cc:1403] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.265s	user 0.012s	sys 0.007s
I20260709 13:17:46.372509 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:46.374421 18245 raft_consensus.cc:359] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.375062 18245 raft_consensus.cc:385] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.375331 18245 raft_consensus.cc:740] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.375319 18297 consensus_queue.cc:1048] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260709 13:17:46.384402 18164 leader_election.cc:343] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.391484 18244 tablet_bootstrap.cc:654] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.377768 18245 consensus_queue.cc:260] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.395177 18245 ts_tablet_manager.cc:1434] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:17:46.396004 18293 consensus_queue.cc:1048] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.396868 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cca6423ee4465cb73609e4e335e66e" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:46.398514 17941 leader_election.cc:304] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:46.400292 18252 raft_consensus.cc:3060] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.401722 18280 raft_consensus.cc:515] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.396315 18245 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:46.412891 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:46.418123 17976 raft_consensus.cc:3060] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.418669 17874 catalog_manager.cc:5697] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.422601 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:46.424193 18355 consensus_queue.cc:1048] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.426192 18247 tablet_bootstrap.cc:654] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.431955 17976 raft_consensus.cc:2468] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:46.415904 18280 leader_election.cc:290] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:46.443900 18164 leader_election.cc:343] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.446512 18246 tablet_bootstrap.cc:654] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.450732 18252 raft_consensus.cc:2749] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.451807 18165 leader_election.cc:304] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:46.453114 18297 raft_consensus.cc:2804] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:46.453651 18297 raft_consensus.cc:697] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:46.454531 18297 consensus_queue.cc:237] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.459386 18245 tablet_bootstrap.cc:654] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.462874 18360 consensus_queue.cc:1048] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.465526 18360 consensus_queue.cc:1048] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.472016 18293 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.472586 18293 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.474659 18293 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:46.475528 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:46.475847 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:46.479764 18089 leader_election.cc:343] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.480046 18360 raft_consensus.cc:493] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:46.485699 18091 leader_election.cc:343] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.486220 18091 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:46.480587 18360 raft_consensus.cc:515] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.489387 18360 leader_election.cc:290] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:46.494565 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:46.496721 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:46.501992 18017 leader_election.cc:304] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:46.502556 18244 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:46.503108 18244 ts_tablet_manager.cc:1403] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.142s	user 0.008s	sys 0.027s
I20260709 13:17:46.504289 18360 raft_consensus.cc:2749] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.492808 18293 raft_consensus.cc:2749] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.515275 18244 raft_consensus.cc:359] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.516189 18244 raft_consensus.cc:385] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.516485 18244 raft_consensus.cc:740] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.523118 17975 raft_consensus.cc:3060] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.525105 18358 consensus_queue.cc:1048] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.518558 18244 consensus_queue.cc:260] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.539700 18244 ts_tablet_manager.cc:1434] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 13:17:46.541031 18244 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:46.544592 18199 raft_consensus.cc:3060] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.544787 18247 tablet_bootstrap.cc:492] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:46.545388 18247 ts_tablet_manager.cc:1403] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.178s	user 0.025s	sys 0.000s
I20260709 13:17:46.548367 18247 raft_consensus.cc:359] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.549158 18247 raft_consensus.cc:385] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.549443 18247 raft_consensus.cc:740] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.550289 18247 consensus_queue.cc:260] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.559764 18247 ts_tablet_manager.cc:1434] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 13:17:46.561611 18247 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:46.567793 18247 tablet_bootstrap.cc:654] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.571103 18297 consensus_queue.cc:1048] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.584977 18297 consensus_queue.cc:1048] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.590617 18246 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:46.591161 18246 ts_tablet_manager.cc:1403] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.226s	user 0.026s	sys 0.025s
I20260709 13:17:46.602447 18246 raft_consensus.cc:359] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.603113 18246 raft_consensus.cc:385] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.603377 18246 raft_consensus.cc:740] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.604125 18246 consensus_queue.cc:260] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.605033 18297 raft_consensus.cc:493] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.605533 18297 raft_consensus.cc:515] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.606380 18355 consensus_queue.cc:1048] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.606690 18244 tablet_bootstrap.cc:654] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.611812 18246 ts_tablet_manager.cc:1434] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260709 13:17:46.612823 18246 tablet_bootstrap.cc:492] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:46.612607 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.614341 18297 leader_election.cc:290] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:46.619285 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd76d33e673a4966a9c91b362fafb0b1" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:46.628957 18165 leader_election.cc:304] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:46.617028 17871 catalog_manager.cc:5697] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.635362 18358 consensus_queue.cc:1048] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.664057 18297 raft_consensus.cc:2749] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:46.666373 18245 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:46.666898 18245 ts_tablet_manager.cc:1403] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.271s	user 0.005s	sys 0.017s
I20260709 13:17:46.667548 18344 consensus_queue.cc:1048] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.668911 18247 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:46.669409 18247 ts_tablet_manager.cc:1403] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.108s	user 0.010s	sys 0.026s
I20260709 13:17:46.669682 18245 raft_consensus.cc:359] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.670359 18245 raft_consensus.cc:385] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.670608 18245 raft_consensus.cc:740] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.674023 18245 consensus_queue.cc:260] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.674737 18246 tablet_bootstrap.cc:654] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.676169 18245 ts_tablet_manager.cc:1434] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 13:17:46.677263 18245 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:46.680480 18247 raft_consensus.cc:359] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.682900 18247 raft_consensus.cc:385] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.683176 18247 raft_consensus.cc:740] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.684929 18125 raft_consensus.cc:3060] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.683853 18247 consensus_queue.cc:260] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.688401 18295 raft_consensus.cc:493] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.689014 18295 raft_consensus.cc:515] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.690479 18244 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:46.691151 18244 ts_tablet_manager.cc:1403] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.150s	user 0.019s	sys 0.022s
I20260709 13:17:46.692023 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26de913e9ce4dad845f7173d82adfc1" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.694563 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26de913e9ce4dad845f7173d82adfc1" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:46.695679 18166 leader_election.cc:343] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.693130 18295 leader_election.cc:290] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:46.696235 18165 leader_election.cc:343] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.696743 18165 leader_election.cc:304] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:46.697167 18244 raft_consensus.cc:359] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.697896 18244 raft_consensus.cc:385] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.697974 18295 raft_consensus.cc:2749] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.705216 18247 ts_tablet_manager.cc:1434] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.002s
I20260709 13:17:46.707222 18360 consensus_queue.cc:1048] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.708492 18245 tablet_bootstrap.cc:654] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.698205 18244 raft_consensus.cc:740] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
W20260709 13:17:46.713435 17941 consensus_peers.cc:597] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:46.722450 18244 consensus_queue.cc:260] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.724752 18246 tablet_bootstrap.cc:492] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:46.727857 18244 ts_tablet_manager.cc:1434] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.002s
I20260709 13:17:46.732136 18244 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:46.730863 18247 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:46.739696 18246 ts_tablet_manager.cc:1403] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.127s	user 0.009s	sys 0.006s
I20260709 13:17:46.748418 18244 tablet_bootstrap.cc:654] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.752652 18246 raft_consensus.cc:359] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.753430 18246 raft_consensus.cc:385] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.753700 18246 raft_consensus.cc:740] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.758803 18247 tablet_bootstrap.cc:654] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.760600 18246 consensus_queue.cc:260] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.762472 18319 consensus_queue.cc:1048] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.765298 18246 ts_tablet_manager.cc:1434] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:17:46.769187 18246 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
W20260709 13:17:46.783766 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:46.784953 18246 tablet_bootstrap.cc:654] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.794487 18051 raft_consensus.cc:3060] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.794905 18244 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:46.803189 18244 ts_tablet_manager.cc:1403] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.071s	user 0.009s	sys 0.009s
I20260709 13:17:46.805691 18247 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:46.806234 18247 ts_tablet_manager.cc:1403] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.076s	user 0.013s	sys 0.004s
I20260709 13:17:46.809121 18245 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:46.809798 18245 ts_tablet_manager.cc:1403] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.133s	user 0.012s	sys 0.016s
I20260709 13:17:46.809849 18247 raft_consensus.cc:359] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.810595 18247 raft_consensus.cc:385] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.812865 18245 raft_consensus.cc:359] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.813701 18245 raft_consensus.cc:385] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.814023 18245 raft_consensus.cc:740] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.814308 18244 raft_consensus.cc:359] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.814970 18244 raft_consensus.cc:385] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.814814 18245 consensus_queue.cc:260] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.815450 18244 raft_consensus.cc:740] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.816563 18244 consensus_queue.cc:260] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.817437 18245 ts_tablet_manager.cc:1434] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:46.818409 18244 ts_tablet_manager.cc:1434] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:17:46.819165 18245 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:46.810878 18247 raft_consensus.cc:740] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.822399 18247 consensus_queue.cc:260] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.829075 18247 ts_tablet_manager.cc:1434] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:17:46.829660 18246 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:46.830178 18246 ts_tablet_manager.cc:1403] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.000s
I20260709 13:17:46.835692 18244 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:46.830091 18247 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:46.840456 18345 raft_consensus.cc:493] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.841009 18345 raft_consensus.cc:515] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.841581 18244 tablet_bootstrap.cc:654] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.841624 18246 raft_consensus.cc:359] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.842435 18246 raft_consensus.cc:385] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.842691 18246 raft_consensus.cc:740] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.846546 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.847777 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:46.849314 17941 leader_election.cc:343] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:46.850272 17943 leader_election.cc:343] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.850752 17943 leader_election.cc:304] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:46.851684 18252 raft_consensus.cc:2749] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.844617 18246 consensus_queue.cc:260] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.854416 18295 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.854904 18295 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.843259 18345 leader_election.cc:290] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.859979 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:46.860863 18247 tablet_bootstrap.cc:654] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:46.861687 18165 leader_election.cc:343] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.862864 18295 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:46.864351 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.865439 18126 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:46.867123 18166 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:46.868462 18245 tablet_bootstrap.cc:654] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.870198 18295 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:46.870922 18295 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.871390 18295 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.882615 18051 raft_consensus.cc:3060] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.885989 18295 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.887846 18280 consensus_queue.cc:1048] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.891613 18319 raft_consensus.cc:493] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.892230 18319 raft_consensus.cc:515] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.894418 18319 leader_election.cc:290] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:46.896930 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.897122 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:46.897853 18126 raft_consensus.cc:2393] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:46.898069 18199 raft_consensus.cc:2393] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:46.899689 18015 leader_election.cc:304] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:46.900942 18319 raft_consensus.cc:3060] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.901796 18246 ts_tablet_manager.cc:1434] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.000s
I20260709 13:17:46.903025 18246 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:46.909474 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:46.910251 18126 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.914041 18319 raft_consensus.cc:2749] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.915666 18280 consensus_queue.cc:1048] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:46.917232 18244 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:46.918365 18244 ts_tablet_manager.cc:1403] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.099s	user 0.038s	sys 0.037s
I20260709 13:17:46.919605 18247 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:46.920275 18247 ts_tablet_manager.cc:1403] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.000s
I20260709 13:17:46.921310 18244 raft_consensus.cc:359] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.922045 18244 raft_consensus.cc:385] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.922308 18244 raft_consensus.cc:740] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.922955 18244 consensus_queue.cc:260] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:46.923614 18247 raft_consensus.cc:359] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.925261 18244 ts_tablet_manager.cc:1434] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:46.926232 18126 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:46.939327 18244 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:46.940073 18247 raft_consensus.cc:385] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.940420 18247 raft_consensus.cc:740] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.940491 18166 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:46.941510 18280 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:46.942154 18280 raft_consensus.cc:697] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:46.943058 18280 consensus_queue.cc:237] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.943969 18246 tablet_bootstrap.cc:654] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.946051 18244 tablet_bootstrap.cc:654] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.946022 18297 raft_consensus.cc:493] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.946648 18297 raft_consensus.cc:515] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.948789 18297 leader_election.cc:290] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.941358 18247 consensus_queue.cc:260] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:46.950910 18247 ts_tablet_manager.cc:1434] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:17:46.953035 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
W20260709 13:17:46.954752 18165 leader_election.cc:343] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.955483 18295 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:46.957717 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:46.960811 18126 raft_consensus.cc:2468] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:46.960090 18244 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:46.951875 18247 tablet_bootstrap.cc:492] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:46.967257 18244 ts_tablet_manager.cc:1403] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.000s
I20260709 13:17:46.967924 18166 leader_election.cc:304] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:46.968999 18295 raft_consensus.cc:493] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.970105 18244 raft_consensus.cc:359] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.970856 18244 raft_consensus.cc:385] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.971127 18244 raft_consensus.cc:740] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.970074 18295 raft_consensus.cc:515] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.971901 18244 consensus_queue.cc:260] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.972764 18247 tablet_bootstrap.cc:654] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.973996 18244 ts_tablet_manager.cc:1434] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:46.974995 18244 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:46.975576 18280 raft_consensus.cc:2804] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:46.975968 18280 raft_consensus.cc:493] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.976329 18280 raft_consensus.cc:3060] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.976643 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:46.977962 18166 leader_election.cc:343] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:46.978874 18295 leader_election.cc:290] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.980597 18244 tablet_bootstrap.cc:654] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:46.983657 18280 raft_consensus.cc:515] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:46.989562 18280 leader_election.cc:290] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:46.987053 18246 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:46.990353 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:46.990374 18246 ts_tablet_manager.cc:1403] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.088s	user 0.010s	sys 0.001s
I20260709 13:17:46.991071 18125 raft_consensus.cc:3060] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.993456 18246 raft_consensus.cc:359] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.994122 18246 raft_consensus.cc:385] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:46.994369 18246 raft_consensus.cc:740] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.995960 18246 consensus_queue.cc:260] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:46.997005 17873 catalog_manager.cc:5697] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.998641 18125 raft_consensus.cc:2468] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:46.999847 18246 ts_tablet_manager.cc:1434] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:17:47.000350 18166 leader_election.cc:304] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:47.001233 18280 raft_consensus.cc:2804] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:47.001441 18245 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:47.001708 18280 raft_consensus.cc:697] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:47.002660 18280 consensus_queue.cc:237] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:47.001770 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 53634.9us
I20260709 13:17:47.004385 18246 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:47.005561 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.006207 18051 raft_consensus.cc:2468] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:47.008342 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.008950 18051 raft_consensus.cc:2468] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:47.010836 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0745baef9dbd4cd0a943445a5aa9cc6b" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:47.015769 18051 raft_consensus.cc:3060] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:47.017187 18164 leader_election.cc:304] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:47.019565 18244 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:47.020077 18244 ts_tablet_manager.cc:1403] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.009s
I20260709 13:17:47.020162 18419 raft_consensus.cc:2804] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.020670 18419 raft_consensus.cc:493] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.021047 18419 raft_consensus.cc:3060] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.022807 18244 raft_consensus.cc:359] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.023520 18244 raft_consensus.cc:385] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.023761 18244 raft_consensus.cc:740] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.017549 18246 tablet_bootstrap.cc:654] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.017933 18245 ts_tablet_manager.cc:1403] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.199s	user 0.023s	sys 0.025s
I20260709 13:17:47.026152 18247 tablet_bootstrap.cc:492] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:47.026650 18247 ts_tablet_manager.cc:1403] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.075s	user 0.020s	sys 0.004s
I20260709 13:17:47.026650 18245 raft_consensus.cc:359] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.027508 18245 raft_consensus.cc:385] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.027782 18245 raft_consensus.cc:740] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.028436 18245 consensus_queue.cc:260] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.029497 18247 raft_consensus.cc:359] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.030128 18247 raft_consensus.cc:385] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.030365 18247 raft_consensus.cc:740] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.030485 18245 ts_tablet_manager.cc:1434] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:17:47.030980 18247 consensus_queue.cc:260] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.031622 18245 tablet_bootstrap.cc:492] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:47.032986 18247 ts_tablet_manager.cc:1434] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:47.037050 18245 tablet_bootstrap.cc:654] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.041935 18244 consensus_queue.cc:260] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.042805 18319 raft_consensus.cc:493] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.043313 18319 raft_consensus.cc:515] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.045253 18319 leader_election.cc:290] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:47.037407 17872 catalog_manager.cc:5697] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 2, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:47.046774 18244 ts_tablet_manager.cc:1434] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.026s	user 0.002s	sys 0.004s
I20260709 13:17:47.047802 18244 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:47.054231 18419 raft_consensus.cc:515] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.055233 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.055763 18244 tablet_bootstrap.cc:654] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.056538 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:47.067827 17977 raft_consensus.cc:2468] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
W20260709 13:17:47.068636 18018 leader_election.cc:343] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.058174 18051 raft_consensus.cc:2468] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:47.073724 18017 leader_election.cc:304] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:47.076298 18319 raft_consensus.cc:2804] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.076817 18319 raft_consensus.cc:493] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.077175 18319 raft_consensus.cc:3060] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.082832 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:47.061529 18247 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:47.087746 18051 raft_consensus.cc:3060] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.094306 18247 tablet_bootstrap.cc:654] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.097295 18319 raft_consensus.cc:515] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.100592 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:47.101231 17977 raft_consensus.cc:3060] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.101897 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
W20260709 13:17:47.103750 18166 leader_election.cc:343] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.106618 18419 leader_election.cc:290] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:47.106871 17976 raft_consensus.cc:3060] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.108417 18246 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:47.108915 18246 ts_tablet_manager.cc:1403] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.108s	user 0.027s	sys 0.002s
I20260709 13:17:47.111683 18246 raft_consensus.cc:359] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.108263 17977 raft_consensus.cc:2468] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:47.112506 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:47.112334 18246 raft_consensus.cc:385] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.113341 18246 raft_consensus.cc:740] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.113867 18017 leader_election.cc:304] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:47.114218 18252 raft_consensus.cc:493] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.114115 18246 consensus_queue.cc:260] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.114758 18252 raft_consensus.cc:515] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.115553 18361 raft_consensus.cc:2804] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.116015 18361 raft_consensus.cc:697] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:47.116592 18246 ts_tablet_manager.cc:1434] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
W20260709 13:17:47.116986 18018 leader_election.cc:343] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.117763 18246 tablet_bootstrap.cc:492] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:47.116103 18245 tablet_bootstrap.cc:492] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:47.119179 18358 raft_consensus.cc:493] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.119475 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.119690 18358 raft_consensus.cc:515] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.120353 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:47.121491 17940 leader_election.cc:343] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.119380 18245 ts_tablet_manager.cc:1403] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.088s	user 0.006s	sys 0.016s
I20260709 13:17:47.122287 18358 leader_election.cc:290] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
W20260709 13:17:47.123020 17941 leader_election.cc:343] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.123591 17941 leader_election.cc:304] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:47.123977 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.124436 18345 raft_consensus.cc:2749] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:47.124706 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.124169 18051 raft_consensus.cc:2468] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:47.116914 18361 consensus_queue.cc:237] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
W20260709 13:17:47.125558 18015 leader_election.cc:343] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.116874 18252 leader_election.cc:290] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:47.126164 18164 leader_election.cc:304] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:47.127336 18419 raft_consensus.cc:2804] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.128105 18419 raft_consensus.cc:697] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:47.128963 18419 consensus_queue.cc:237] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.129801 18246 tablet_bootstrap.cc:654] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.130329 18245 raft_consensus.cc:359] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.130976 18245 raft_consensus.cc:385] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:47.130136 18018 leader_election.cc:343] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.131264 18245 raft_consensus.cc:740] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.132207 18245 consensus_queue.cc:260] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.133026 18244 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:47.133541 18244 ts_tablet_manager.cc:1403] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.008s
I20260709 13:17:47.134294 18245 ts_tablet_manager.cc:1434] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
I20260709 13:17:47.135313 18245 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:47.136392 18244 raft_consensus.cc:359] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.137106 18244 raft_consensus.cc:385] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.137382 18244 raft_consensus.cc:740] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.138144 18244 consensus_queue.cc:260] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.112370 18319 leader_election.cc:290] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:47.139056 18018 leader_election.cc:304] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:47.141531 18244 ts_tablet_manager.cc:1434] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:17:47.142836 18244 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:47.143993 18358 raft_consensus.cc:2749] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:47.147511 18245 tablet_bootstrap.cc:654] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.153484 18244 tablet_bootstrap.cc:654] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.156217 18252 raft_consensus.cc:493] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.156767 18252 raft_consensus.cc:515] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.159067 18245 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:47.160545 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.159577 18245 ts_tablet_manager.cc:1403] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 13:17:47.161274 18199 raft_consensus.cc:2468] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:47.162977 17940 leader_election.cc:304] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:47.164284 18345 raft_consensus.cc:2804] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.164711 18345 raft_consensus.cc:493] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.165046 18345 raft_consensus.cc:3060] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.158852 17874 catalog_manager.cc:5697] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:47.166818 18245 raft_consensus.cc:359] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.167244 18358 raft_consensus.cc:493] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.167542 18245 raft_consensus.cc:385] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.169482 18245 raft_consensus.cc:740] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.170063 18358 raft_consensus.cc:515] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.171089 18245 consensus_queue.cc:260] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.171423 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.172299 18358 leader_election.cc:290] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:47.173609 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:47.175460 17941 leader_election.cc:343] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.158876 18252 leader_election.cc:290] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:47.178771 18245 ts_tablet_manager.cc:1434] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:17:47.179039 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:47.180027 18245 tablet_bootstrap.cc:492] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:47.184687 17871 catalog_manager.cc:5697] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.185789 18345 raft_consensus.cc:515] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.189419 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:47.190186 18199 raft_consensus.cc:3060] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.191004 18017 leader_election.cc:304] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:47.192461 18319 raft_consensus.cc:2749] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:47.196787 18245 tablet_bootstrap.cc:654] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.200690 18199 raft_consensus.cc:2468] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:47.202425 17940 leader_election.cc:304] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:47.203223 18346 raft_consensus.cc:2804] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.203732 18346 raft_consensus.cc:697] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:47.204505 18346 consensus_queue.cc:237] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.211038 18051 raft_consensus.cc:3060] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.233883 18345 leader_election.cc:290] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:47.238641 18246 tablet_bootstrap.cc:492] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:47.239521 18246 ts_tablet_manager.cc:1403] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.122s	user 0.008s	sys 0.008s
I20260709 13:17:47.244284 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:47.246387 18246 raft_consensus.cc:359] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.247967 18245 tablet_bootstrap.cc:492] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:47.248479 18245 ts_tablet_manager.cc:1403] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.069s	user 0.025s	sys 0.013s
W20260709 13:17:47.249332 17941 leader_election.cc:343] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.251312 18245 raft_consensus.cc:359] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.252017 18245 raft_consensus.cc:385] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.252260 18245 raft_consensus.cc:740] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.252851 18245 consensus_queue.cc:260] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.253995 18246 raft_consensus.cc:385] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.254415 18246 raft_consensus.cc:740] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.255131 18245 ts_tablet_manager.cc:1434] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:47.255473 18246 consensus_queue.cc:260] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.256173 18245 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:47.260649 18247 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:47.261257 18247 ts_tablet_manager.cc:1403] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.200s	user 0.007s	sys 0.035s
I20260709 13:17:47.262498 18246 ts_tablet_manager.cc:1434] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.002s
I20260709 13:17:47.264175 18247 raft_consensus.cc:359] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.264915 18247 raft_consensus.cc:385] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.265178 18247 raft_consensus.cc:740] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.264222 18345 raft_consensus.cc:493] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.266880 18419 raft_consensus.cc:493] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.266364 18345 raft_consensus.cc:515] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.267346 18419 raft_consensus.cc:515] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.266749 18247 consensus_queue.cc:260] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.269515 18419 leader_election.cc:290] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:47.271158 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7187dfc4efc34078a81a1f59637521c8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.271972 18126 raft_consensus.cc:2468] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:47.272184 18345 leader_election.cc:290] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.274084 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7187dfc4efc34078a81a1f59637521c8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.276624 18245 tablet_bootstrap.cc:654] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.277177 17943 leader_election.cc:304] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:47.271296 18247 ts_tablet_manager.cc:1434] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
W20260709 13:17:47.278566 17940 leader_election.cc:343] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.279019 18345 raft_consensus.cc:2804] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.279381 18345 raft_consensus.cc:493] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.278676 18247 tablet_bootstrap.cc:492] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:47.279870 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1606bb8512a4c5ba5133be786298984" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.280402 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1606bb8512a4c5ba5133be786298984" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:47.281159 18164 leader_election.cc:343] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.265971 18246 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:47.281042 17976 raft_consensus.cc:2468] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:47.279814 18345 raft_consensus.cc:3060] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.285069 18165 leader_election.cc:304] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:47.286198 18419 raft_consensus.cc:2804] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.286571 18419 raft_consensus.cc:493] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.286893 18419 raft_consensus.cc:3060] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.262501 17871 catalog_manager.cc:5697] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.288475 18246 tablet_bootstrap.cc:654] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.292752 18244 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:47.293236 18244 ts_tablet_manager.cc:1403] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.151s	user 0.006s	sys 0.007s
I20260709 13:17:47.295064 18419 raft_consensus.cc:515] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.295960 18244 raft_consensus.cc:359] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.296617 18244 raft_consensus.cc:385] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.298141 18419 leader_election.cc:290] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:47.296888 18244 raft_consensus.cc:740] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.300519 18244 consensus_queue.cc:260] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.301476 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1606bb8512a4c5ba5133be786298984" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:47.302071 17976 raft_consensus.cc:3060] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.302312 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1606bb8512a4c5ba5133be786298984" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
W20260709 13:17:47.303722 18164 leader_election.cc:343] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.304276 18244 ts_tablet_manager.cc:1434] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:17:47.305294 18244 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:47.314507 18247 tablet_bootstrap.cc:654] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.325119 18244 tablet_bootstrap.cc:654] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.325805 18246 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:47.326283 18246 ts_tablet_manager.cc:1403] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.003s
I20260709 13:17:47.328639 18296 raft_consensus.cc:493] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.328351 18345 raft_consensus.cc:515] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.329121 18296 raft_consensus.cc:515] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.328886 18246 raft_consensus.cc:359] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.331235 18246 raft_consensus.cc:385] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.331094 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7187dfc4efc34078a81a1f59637521c8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:47.331634 18246 raft_consensus.cc:740] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.331882 18126 raft_consensus.cc:3060] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.332366 18246 consensus_queue.cc:260] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.332237 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26de913e9ce4dad845f7173d82adfc1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:47.332917 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26de913e9ce4dad845f7173d82adfc1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.333627 18199 raft_consensus.cc:2468] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:47.334626 18246 ts_tablet_manager.cc:1434] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:47.334789 18089 leader_election.cc:304] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:47.335639 18246 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:47.337834 17976 raft_consensus.cc:2468] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:47.331132 18296 leader_election.cc:290] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:47.338676 18296 raft_consensus.cc:2804] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.339001 18296 raft_consensus.cc:493] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.339294 18296 raft_consensus.cc:3060] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:47.340442 18091 leader_election.cc:343] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.342423 18245 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:47.343129 18245 ts_tablet_manager.cc:1403] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.002s
I20260709 13:17:47.342555 18165 leader_election.cc:304] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:47.344513 18345 leader_election.cc:290] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.344861 18419 raft_consensus.cc:2804] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.345263 18419 raft_consensus.cc:697] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:47.345721 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7187dfc4efc34078a81a1f59637521c8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:47.345877 18126 raft_consensus.cc:2468] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:47.346370 18199 raft_consensus.cc:3060] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.346062 18419 consensus_queue.cc:237] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.347252 17943 leader_election.cc:304] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:47.350240 18293 raft_consensus.cc:493] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.350728 18293 raft_consensus.cc:515] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.353297 18247 tablet_bootstrap.cc:492] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:47.353765 18247 ts_tablet_manager.cc:1403] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.000s
I20260709 13:17:47.354069 18245 raft_consensus.cc:359] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.354710 18245 raft_consensus.cc:385] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.355023 18245 raft_consensus.cc:740] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.355677 18245 consensus_queue.cc:260] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.357649 18247 raft_consensus.cc:359] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.358651 18246 tablet_bootstrap.cc:654] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.358850 18247 raft_consensus.cc:385] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.359179 18247 raft_consensus.cc:740] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.359881 18346 raft_consensus.cc:493] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.360188 18247 consensus_queue.cc:260] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.360510 18346 raft_consensus.cc:515] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.361090 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3918cfe707c24309bd3a731cca5bcbef" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.361691 18051 raft_consensus.cc:2468] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:47.362056 18247 ts_tablet_manager.cc:1434] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:47.362706 18346 leader_election.cc:290] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:47.362960 18247 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:47.362908 18090 leader_election.cc:304] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:47.357203 18345 raft_consensus.cc:2804] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.363735 18295 raft_consensus.cc:493] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.364321 18295 raft_consensus.cc:515] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.365079 18345 raft_consensus.cc:697] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:47.364573 18293 leader_election.cc:290] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.365669 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3918cfe707c24309bd3a731cca5bcbef" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.366241 18295 leader_election.cc:290] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:47.366238 18345 consensus_queue.cc:237] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.367961 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.367988 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.369520 18244 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:47.370034 18244 ts_tablet_manager.cc:1403] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.065s	user 0.005s	sys 0.006s
I20260709 13:17:47.369541 18125 raft_consensus.cc:2468] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
W20260709 13:17:47.371968 17941 leader_election.cc:343] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.354599 18296 raft_consensus.cc:515] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.372006 18199 raft_consensus.cc:2468] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:47.368050 18200 raft_consensus.cc:2468] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:47.374998 18166 leader_election.cc:304] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:47.376399 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.376602 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:47.377679 18165 leader_election.cc:343] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.378468 18244 raft_consensus.cc:359] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.379128 18244 raft_consensus.cc:385] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.379365 18244 raft_consensus.cc:740] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.380054 18244 consensus_queue.cc:260] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.382647 18355 raft_consensus.cc:2804] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.383134 18355 raft_consensus.cc:493] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.383520 18355 raft_consensus.cc:3060] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:47.388079 17943 leader_election.cc:343] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.388617 17943 leader_election.cc:304] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:47.389542 18295 raft_consensus.cc:2804] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.390069 18295 raft_consensus.cc:493] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.390411 18295 raft_consensus.cc:3060] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.390868 18296 leader_election.cc:290] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:47.391881 17976 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.392280 18245 ts_tablet_manager.cc:1434] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 13:17:47.392758 18244 ts_tablet_manager.cc:1434] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:17:47.404774 18244 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:47.389673 18346 raft_consensus.cc:2749] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:47.414973 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26de913e9ce4dad845f7173d82adfc1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
W20260709 13:17:47.413540 18090 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 50720.2us
I20260709 13:17:47.415900 18293 raft_consensus.cc:493] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.422286 18293 raft_consensus.cc:515] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.423527 18199 raft_consensus.cc:3060] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.428216 17871 catalog_manager.cc:5697] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.431901 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26de913e9ce4dad845f7173d82adfc1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:47.432930 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:47.433724 17975 raft_consensus.cc:2393] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
W20260709 13:17:47.433871 18091 leader_election.cc:343] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.436432 18247 tablet_bootstrap.cc:654] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.429774 18293 leader_election.cc:290] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:47.443542 18126 raft_consensus.cc:1275] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:47.445106 18280 consensus_queue.cc:1048] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:47.446583 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "127bfde2c3624e5bb2d0f68929889394" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.444983 18245 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:47.463104 18246 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:47.463680 18246 ts_tablet_manager.cc:1403] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.128s	user 0.028s	sys 0.052s
W20260709 13:17:47.452857 17940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 43931.9us
I20260709 13:17:47.443328 18355 raft_consensus.cc:515] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.459931 18344 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.469724 18199 raft_consensus.cc:2468] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:47.457557 18090 leader_election.cc:304] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:47.473351 18089 leader_election.cc:304] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:47.476195 18296 raft_consensus.cc:3060] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.476439 18247 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:47.476962 18247 ts_tablet_manager.cc:1403] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.012s
I20260709 13:17:47.470305 18280 consensus_queue.cc:1048] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:17:47.470521 18344 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.482576 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3918cfe707c24309bd3a731cca5bcbef" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:47.483150 18246 raft_consensus.cc:359] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.483834 18246 raft_consensus.cc:385] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.484071 18246 raft_consensus.cc:740] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.484784 18246 consensus_queue.cc:260] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.479954 17872 catalog_manager.cc:5697] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:47.474748 18293 raft_consensus.cc:2804] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.483175 18200 raft_consensus.cc:3060] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.489838 18355 leader_election.cc:290] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.490408 18246 ts_tablet_manager.cc:1434] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:17:47.497370 18246 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:47.497256 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.497503 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.498107 18125 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:47.499266 17943 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:47.496248 18344 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.504526 18244 tablet_bootstrap.cc:654] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.508163 18200 raft_consensus.cc:2468] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:47.509528 18089 leader_election.cc:304] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:47.510123 18246 tablet_bootstrap.cc:654] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.517517 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3918cfe707c24309bd3a731cca5bcbef" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:47.518189 18052 raft_consensus.cc:3060] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.525426 18052 raft_consensus.cc:2468] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:47.529112 18446 raft_consensus.cc:2804] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.529578 18446 raft_consensus.cc:697] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:47.530423 18446 consensus_queue.cc:237] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.537295 18293 raft_consensus.cc:697] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:47.538497 18293 consensus_queue.cc:237] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.488425 18295 raft_consensus.cc:515] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.553929 18361 consensus_queue.cc:1048] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:47.555687 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:47.555991 18296 raft_consensus.cc:2749] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:47.556371 18124 raft_consensus.cc:3060] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.558203 17974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:47.553859 18295 leader_election.cc:290] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:47.559468 18245 tablet_bootstrap.cc:654] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:47.561558 18165 leader_election.cc:343] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.562171 18355 raft_consensus.cc:493] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.562585 18280 consensus_queue.cc:1048] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:47.563830 18346 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:47.562669 18355 raft_consensus.cc:515] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:47.565783 18346 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc8a9ec31a5745d69dae4262b271e568)
I20260709 13:17:47.566210 18346 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:47.567062 18355 leader_election.cc:290] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.570688 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.572392 18052 raft_consensus.cc:2468] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:47.572849 18247 raft_consensus.cc:359] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.573505 18247 raft_consensus.cc:385] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.573693 18090 leader_election.cc:304] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:47.574468 18124 raft_consensus.cc:2468] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:47.576236 18166 leader_election.cc:304] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:47.574746 18355 raft_consensus.cc:2804] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.578121 18355 raft_consensus.cc:493] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.578552 18355 raft_consensus.cc:3060] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.578996 18295 raft_consensus.cc:2804] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:47.573761 18247 raft_consensus.cc:740] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.584761 18247 consensus_queue.cc:260] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.592595 18247 ts_tablet_manager.cc:1434] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.115s	user 0.006s	sys 0.001s
I20260709 13:17:47.588430 18295 raft_consensus.cc:697] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:47.604868 18247 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
W20260709 13:17:47.587082 18089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 76484us
I20260709 13:17:47.628968 18244 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:47.629509 18244 ts_tablet_manager.cc:1403] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.225s	user 0.017s	sys 0.002s
I20260709 13:17:47.617342 18346 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.630779 18361 raft_consensus.cc:493] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.643589 18051 raft_consensus.cc:3060] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.633899 18295 consensus_queue.cc:237] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.648916 18246 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:47.655515 18246 ts_tablet_manager.cc:1403] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.158s	user 0.018s	sys 0.009s
I20260709 13:17:47.656823 18247 tablet_bootstrap.cc:654] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.657061 18244 raft_consensus.cc:359] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.659443 18244 raft_consensus.cc:385] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.659777 18244 raft_consensus.cc:740] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.655853 18361 raft_consensus.cc:515] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.673271 18461 raft_consensus.cc:493] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.670990 18244 consensus_queue.cc:260] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.678295 18244 ts_tablet_manager.cc:1434] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
W20260709 13:17:47.672592 18166 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 46475.2us
I20260709 13:17:47.695569 18344 raft_consensus.cc:493] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.696105 18344 raft_consensus.cc:515] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.696223 18360 consensus_queue.cc:1048] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:47.698151 18344 leader_election.cc:290] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.700210 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.700357 18346 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.701030 18123 raft_consensus.cc:2393] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:47.701553 18361 leader_election.cc:290] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:47.702816 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:47.703452 17977 raft_consensus.cc:2468] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:47.706774 18017 leader_election.cc:304] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:47.707719 18358 raft_consensus.cc:2804] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:47.708098 18358 raft_consensus.cc:493] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.708433 18358 raft_consensus.cc:3060] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.676883 18461 raft_consensus.cc:515] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.698083 18246 raft_consensus.cc:359] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.711907 18246 raft_consensus.cc:385] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.712153 18246 raft_consensus.cc:740] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.713143 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.708491 18244 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:47.719772 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:47.717873 18355 raft_consensus.cc:515] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:47.754645 18166 consensus_peers.cc:597] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.754199 18358 raft_consensus.cc:515] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.735268 18461 leader_election.cc:290] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:47.786156 18247 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:47.781697 18246 consensus_queue.cc:260] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.786927 18247 ts_tablet_manager.cc:1403] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.186s	user 0.011s	sys 0.006s
I20260709 13:17:47.779844 17976 raft_consensus.cc:1240] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Rejecting Update request from peer fc8a9ec31a5745d69dae4262b271e568 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:47.773933 18344 raft_consensus.cc:493] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.798620 18052 raft_consensus.cc:1275] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:47.796881 18344 raft_consensus.cc:515] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.801222 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:47.802068 17975 raft_consensus.cc:3060] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.801357 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:47.801193 18246 ts_tablet_manager.cc:1434] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.145s	user 0.011s	sys 0.000s
I20260709 13:17:47.788782 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.808419 17871 catalog_manager.cc:5697] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.814363 18246 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:47.805640 18358 leader_election.cc:290] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:47.816519 18245 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:47.816885 18245 ts_tablet_manager.cc:1403] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.405s	user 0.031s	sys 0.018s
I20260709 13:17:47.819027 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:47.819713 18198 raft_consensus.cc:2468] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:47.796684 18346 consensus_queue.cc:1048] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
W20260709 13:17:47.823289 18089 leader_election.cc:343] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.827694 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:47.827800 17940 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
W20260709 13:17:47.829162 17943 leader_election.cc:343] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.815636 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.841621 17874 catalog_manager.cc:5697] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.826944 18344 leader_election.cc:290] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:47.848405 18244 tablet_bootstrap.cc:654] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.853729 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:47.854393 18198 raft_consensus.cc:3060] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.844698 18447 raft_consensus.cc:2749] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:47.851248 18051 raft_consensus.cc:2393] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc8a9ec31a5745d69dae4262b271e568 in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:47.856920 17874 catalog_manager.cc:5697] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.837906 17940 leader_election.cc:304] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:47.864085 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:47.864836 18052 raft_consensus.cc:3060] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.866818 18461 consensus_queue.cc:1048] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:47.868362 18247 raft_consensus.cc:359] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.869119 18247 raft_consensus.cc:385] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.869388 18247 raft_consensus.cc:740] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.870113 18247 consensus_queue.cc:260] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:47.882328 18199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:47.886761 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:47.888938 18461 consensus_queue.cc:1059] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:47.896286 18089 leader_election.cc:343] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:47.898469 17940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 62633.9us
I20260709 13:17:47.902851 18252 raft_consensus.cc:3060] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.907609 18447 consensus_queue.cc:1048] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:47.913959 18245 raft_consensus.cc:359] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.914798 18245 raft_consensus.cc:385] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:47.915074 18245 raft_consensus.cc:740] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:47.915885 18245 consensus_queue.cc:260] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:47.919029 18280 consensus_queue.cc:1048] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:47.922402 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0771f9326ce4105929b0fca4a301e1e" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:47.923506 18458 raft_consensus.cc:3055] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:47.923926 18458 raft_consensus.cc:740] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:47.924654 18458 consensus_queue.cc:260] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:47.925729 18458 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:47.927331 18346 consensus_queue.cc:1048] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:47.929803 18165 leader_election.cc:304] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
W20260709 13:17:47.931700 17941 leader_election.cc:343] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:47.932178 17941 leader_election.cc:304] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:47.938752 18051 raft_consensus.cc:3060] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:47.942727 18247 ts_tablet_manager.cc:1434] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.155s	user 0.007s	sys 0.000s
I20260709 13:17:47.957794 18252 raft_consensus.cc:2749] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:47.959221 18246 tablet_bootstrap.cc:654] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:47.901734 18355 leader_election.cc:290] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:47.950665 18447 raft_consensus.cc:2749] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:47.911121 17975 raft_consensus.cc:2468] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:47.966379 18245 ts_tablet_manager.cc:1434] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.149s	user 0.006s	sys 0.002s
I20260709 13:17:47.972914 18247 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:47.975634 18245 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:47.966930 18461 consensus_queue.cc:1048] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:17:47.984735 18017 leader_election.cc:304] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:47.996493 18355 consensus_queue.cc:1048] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:48.011381 18358 raft_consensus.cc:2804] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:48.034991 18280 consensus_queue.cc:1048] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:48.029666 18052 raft_consensus.cc:2468] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:48.042871 18447 consensus_queue.cc:1048] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:48.034927 18247 tablet_bootstrap.cc:654] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.046662 18245 tablet_bootstrap.cc:654] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.047437 18244 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:48.048002 18244 ts_tablet_manager.cc:1403] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.340s	user 0.025s	sys 0.026s
I20260709 13:17:48.053006 18198 raft_consensus.cc:2468] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:48.058319 18295 raft_consensus.cc:2749] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:48.048846 18358 raft_consensus.cc:697] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:48.069314 18244 raft_consensus.cc:359] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.071807 18244 raft_consensus.cc:385] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.072046 18244 raft_consensus.cc:740] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.075704 18244 consensus_queue.cc:260] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.093024 18244 ts_tablet_manager.cc:1434] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.002s
I20260709 13:17:48.098738 18358 consensus_queue.cc:237] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.100019 18244 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:17:48.104876 18091 consensus_peers.cc:597] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:48.122160 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:48.130203 18245 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:48.130908 18245 ts_tablet_manager.cc:1403] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.156s	user 0.007s	sys 0.012s
I20260709 13:17:48.154115 18246 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:48.154659 18090 leader_election.cc:304] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:48.156031 18458 raft_consensus.cc:493] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.156545 18458 raft_consensus.cc:515] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.158574 18458 leader_election.cc:290] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:48.159660 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:48.160504 18245 raft_consensus.cc:359] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
W20260709 13:17:48.161095 18164 leader_election.cc:343] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.161164 18245 raft_consensus.cc:385] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.161947 18245 raft_consensus.cc:740] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.162678 18245 consensus_queue.cc:260] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.164795 18245 ts_tablet_manager.cc:1434] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.001s
I20260709 13:17:48.165720 18245 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:48.154706 18246 ts_tablet_manager.cc:1403] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.343s	user 0.012s	sys 0.026s
I20260709 13:17:48.181142 18244 tablet_bootstrap.cc:654] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.182107 18247 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:48.182602 18247 ts_tablet_manager.cc:1403] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.222s	user 0.018s	sys 0.029s
I20260709 13:17:48.183221 18295 consensus_queue.cc:1048] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:48.190999 18456 consensus_queue.cc:1048] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:48.187350 18355 raft_consensus.cc:2804] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:48.192796 18355 raft_consensus.cc:697] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:48.193645 18355 consensus_queue.cc:237] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.194382 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:48.196166 18246 raft_consensus.cc:359] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.198691 18246 raft_consensus.cc:385] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.198993 18246 raft_consensus.cc:740] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
W20260709 13:17:48.196602 18165 leader_election.cc:343] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.200017 18246 consensus_queue.cc:260] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.229341 18246 ts_tablet_manager.cc:1434] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
W20260709 13:17:48.227682 18091 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 45887.1us
I20260709 13:17:48.217435 18165 leader_election.cc:304] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:48.243579 18459 raft_consensus.cc:2749] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:48.243746 18165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 47187.6us
I20260709 13:17:48.317446 18246 tablet_bootstrap.cc:492] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:48.253361 18247 raft_consensus.cc:359] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.318584 18247 raft_consensus.cc:385] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.319015 18247 raft_consensus.cc:740] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.323225 18244 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:48.324414 18245 tablet_bootstrap.cc:654] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.321656 18247 consensus_queue.cc:260] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.323724 18244 ts_tablet_manager.cc:1403] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.224s	user 0.019s	sys 0.011s
I20260709 13:17:48.339286 18247 ts_tablet_manager.cc:1434] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.156s	user 0.011s	sys 0.000s
I20260709 13:17:48.346302 18247 tablet_bootstrap.cc:492] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:48.345271 18252 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.347620 18252 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.349823 18252 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:48.362349 18494 consensus_queue.cc:1048] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:17:48.351390 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:48.363775 18125 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:48.368129 17943 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:48.369099 18489 raft_consensus.cc:2804] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:48.369475 18489 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.369773 18489 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.377048 18244 raft_consensus.cc:359] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.379832 18244 raft_consensus.cc:385] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.380040 18244 raft_consensus.cc:740] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.382071 18244 consensus_queue.cc:260] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.384892 18244 ts_tablet_manager.cc:1434] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.001s
I20260709 13:17:48.388955 18244 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:48.389284 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:48.389922 18050 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:48.394642 18244 tablet_bootstrap.cc:654] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.394960 18246 tablet_bootstrap.cc:654] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.400689 18247 tablet_bootstrap.cc:654] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.402104 17871 catalog_manager.cc:5697] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:48.403568 18490 log.cc:927] Time spent T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:17:48.444689 18489 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.453182 18489 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:48.454869 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:48.455682 18124 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.458559 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:48.461885 18295 consensus_queue.cc:1048] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:48.459420 18049 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.459275 17874 catalog_manager.cc:5697] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.464411 18124 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:48.465657 17943 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:48.468050 18488 raft_consensus.cc:2804] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:48.468559 18488 raft_consensus.cc:697] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:48.469357 18488 consensus_queue.cc:237] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.474807 18245 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:48.475337 18245 ts_tablet_manager.cc:1403] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.310s	user 0.014s	sys 0.002s
I20260709 13:17:48.490029 18244 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:48.490593 18244 ts_tablet_manager.cc:1403] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.104s	user 0.024s	sys 0.032s
I20260709 13:17:48.490271 18245 raft_consensus.cc:359] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.490988 18245 raft_consensus.cc:385] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.492049 18246 tablet_bootstrap.cc:492] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:48.492527 18246 ts_tablet_manager.cc:1403] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.252s	user 0.018s	sys 0.001s
I20260709 13:17:48.493280 18244 raft_consensus.cc:359] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.493935 18244 raft_consensus.cc:385] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.494170 18244 raft_consensus.cc:740] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.494808 18244 consensus_queue.cc:260] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.487358 18049 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:48.497344 18244 ts_tablet_manager.cc:1434] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:17:48.498332 18244 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:48.491325 18245 raft_consensus.cc:740] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.500602 18245 consensus_queue.cc:260] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.504320 18247 tablet_bootstrap.cc:492] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:48.503722 18246 raft_consensus.cc:359] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.504981 18246 raft_consensus.cc:385] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.505172 18244 tablet_bootstrap.cc:654] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.505337 18246 raft_consensus.cc:740] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.511133 18245 ts_tablet_manager.cc:1434] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 13:17:48.512423 18245 tablet_bootstrap.cc:492] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:48.513815 18246 consensus_queue.cc:260] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.516394 18246 ts_tablet_manager.cc:1434] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:17:48.517452 18246 tablet_bootstrap.cc:492] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:48.504813 18247 ts_tablet_manager.cc:1403] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.159s	user 0.016s	sys 0.000s
I20260709 13:17:48.522455 18247 raft_consensus.cc:359] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.531641 18246 tablet_bootstrap.cc:654] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.536474 18488 raft_consensus.cc:493] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.523236 18247 raft_consensus.cc:385] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.537216 18488 raft_consensus.cc:515] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.541368 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:48.541960 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:48.542877 18200 raft_consensus.cc:2468] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
W20260709 13:17:48.543689 17943 leader_election.cc:343] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.545030 17940 leader_election.cc:304] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:48.541106 18488 leader_election.cc:290] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:48.546108 18489 raft_consensus.cc:2804] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:48.546556 18489 raft_consensus.cc:493] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.546906 18489 raft_consensus.cc:3060] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.547516 18245 tablet_bootstrap.cc:654] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.549463 18247 raft_consensus.cc:740] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.550271 18247 consensus_queue.cc:260] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.552778 18247 ts_tablet_manager.cc:1434] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:17:48.553915 18247 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:48.558070 18244 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:48.558641 18244 ts_tablet_manager.cc:1403] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.061s	user 0.036s	sys 0.017s
I20260709 13:17:48.561676 18244 raft_consensus.cc:359] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.562695 18244 raft_consensus.cc:385] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.562980 18244 raft_consensus.cc:740] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.563534 18247 tablet_bootstrap.cc:654] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.563764 18244 consensus_queue.cc:260] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.567031 18245 tablet_bootstrap.cc:492] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:48.567201 18244 ts_tablet_manager.cc:1434] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:17:48.567574 18245 ts_tablet_manager.cc:1403] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.005s
I20260709 13:17:48.568207 18244 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:48.570207 18245 raft_consensus.cc:359] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.570869 18245 raft_consensus.cc:385] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.572643 18245 raft_consensus.cc:740] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.573356 18245 consensus_queue.cc:260] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.575587 18245 ts_tablet_manager.cc:1434] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:17:48.576594 18245 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:48.579126 18244 tablet_bootstrap.cc:654] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.580973 18489 raft_consensus.cc:515] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.592787 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
W20260709 13:17:48.594525 17943 leader_election.cc:343] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.597622 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:48.598345 18200 raft_consensus.cc:3060] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.609429 18198 raft_consensus.cc:3060] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.615294 17875 catalog_manager.cc:5697] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.582245 18245 tablet_bootstrap.cc:654] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.588169 18247 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:48.621155 18247 ts_tablet_manager.cc:1403] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.000s
I20260709 13:17:48.623600 18500 consensus_queue.cc:1048] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:48.624225 18247 raft_consensus.cc:359] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.624884 18247 raft_consensus.cc:385] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.625131 18247 raft_consensus.cc:740] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.625734 18247 consensus_queue.cc:260] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.628576 18500 raft_consensus.cc:493] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.628778 18200 raft_consensus.cc:2468] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:48.631866 18500 raft_consensus.cc:515] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.634932 18500 leader_election.cc:290] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:48.635553 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:48.636229 17977 raft_consensus.cc:2468] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:48.637773 18091 leader_election.cc:304] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:48.639130 18500 raft_consensus.cc:2804] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:48.639503 18500 raft_consensus.cc:493] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.639835 18500 raft_consensus.cc:3060] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.643805 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:48.645550 17940 leader_election.cc:304] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:48.629977 18489 leader_election.cc:290] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:48.646920 18489 raft_consensus.cc:2804] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:48.647188 18498 consensus_queue.cc:1048] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:17:48.648026 18090 leader_election.cc:343] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.649869 18247 ts_tablet_manager.cc:1434] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:17:48.650835 18247 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:48.651876 18293 raft_consensus.cc:493] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.652371 18293 raft_consensus.cc:515] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.654398 18293 leader_election.cc:290] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:48.655040 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:48.654853 18500 raft_consensus.cc:515] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.655741 17977 raft_consensus.cc:2468] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:48.657383 18091 leader_election.cc:304] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:48.657562 18245 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:48.658185 18245 ts_tablet_manager.cc:1403] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.082s	user 0.010s	sys 0.000s
I20260709 13:17:48.658738 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:48.660161 18090 leader_election.cc:343] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.661024 18293 raft_consensus.cc:2804] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:48.661401 18293 raft_consensus.cc:493] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.661727 18293 raft_consensus.cc:3060] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.662448 18245 raft_consensus.cc:359] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.663252 18245 raft_consensus.cc:385] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.663926 18245 raft_consensus.cc:740] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.665292 18245 consensus_queue.cc:260] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.667537 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:48.668076 17977 raft_consensus.cc:3060] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.671545 18498 raft_consensus.cc:493] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.672040 18498 raft_consensus.cc:515] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.674068 18498 leader_election.cc:290] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:48.674865 17977 raft_consensus.cc:2468] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:48.675349 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:48.676945 18015 leader_election.cc:343] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.679527 18246 tablet_bootstrap.cc:492] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:48.680022 18246 ts_tablet_manager.cc:1403] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.163s	user 0.006s	sys 0.006s
I20260709 13:17:48.681504 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:48.681926 18500 leader_election.cc:290] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:48.682992 18090 leader_election.cc:343] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.683279 18293 raft_consensus.cc:515] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.685453 18293 leader_election.cc:290] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:48.686527 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:48.688390 18091 leader_election.cc:304] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
W20260709 13:17:48.688484 18090 leader_election.cc:343] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.689797 18247 tablet_bootstrap.cc:654] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.689908 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:48.691202 18018 leader_election.cc:343] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.691622 18018 leader_election.cc:304] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:48.692049 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:48.692632 17977 raft_consensus.cc:3060] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.695088 18360 raft_consensus.cc:2749] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:48.699798 18489 raft_consensus.cc:697] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:48.700695 18489 consensus_queue.cc:237] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.707818 18245 ts_tablet_manager.cc:1434] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.049s	user 0.004s	sys 0.002s
I20260709 13:17:48.708459 18293 raft_consensus.cc:2804] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:48.708976 18293 raft_consensus.cc:697] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:48.710470 18498 consensus_queue.cc:1048] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:48.719062 18453 consensus_queue.cc:1048] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:48.720646 18246 raft_consensus.cc:359] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.721359 18246 raft_consensus.cc:385] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.721611 18246 raft_consensus.cc:740] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.727679 18246 consensus_queue.cc:260] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.731362 18244 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:48.731925 18244 ts_tablet_manager.cc:1403] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.164s	user 0.014s	sys 0.039s
I20260709 13:17:48.734673 18244 raft_consensus.cc:359] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.735299 18244 raft_consensus.cc:385] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.735565 18244 raft_consensus.cc:740] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.736263 18244 consensus_queue.cc:260] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.739252 18244 ts_tablet_manager.cc:1434] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:48.739452 18246 ts_tablet_manager.cc:1434] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
I20260709 13:17:48.740177 18244 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:48.740432 18246 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:48.743486 18293 consensus_queue.cc:237] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.744836 18245 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:48.750967 17977 raft_consensus.cc:2468] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:48.752388 18091 leader_election.cc:304] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:48.753417 18453 raft_consensus.cc:2804] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:48.754077 17874 catalog_manager.cc:5697] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:48.755265 18453 raft_consensus.cc:697] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:48.756007 18245 tablet_bootstrap.cc:654] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.756322 18453 consensus_queue.cc:237] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.760222 18246 tablet_bootstrap.cc:654] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.765232 18244 tablet_bootstrap.cc:654] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.805802 17874 catalog_manager.cc:5697] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:48.823230 18244 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:48.823803 18244 ts_tablet_manager.cc:1403] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.084s	user 0.005s	sys 0.012s
I20260709 13:17:48.826218 18489 raft_consensus.cc:493] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.826371 18244 raft_consensus.cc:359] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.826715 18489 raft_consensus.cc:515] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.830015 18246 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:48.830498 18246 ts_tablet_manager.cc:1403] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.090s	user 0.007s	sys 0.024s
I20260709 13:17:48.829852 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:48.832834 18245 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
W20260709 13:17:48.832930 17940 leader_election.cc:343] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.833326 18245 ts_tablet_manager.cc:1403] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.114s	user 0.015s	sys 0.015s
I20260709 13:17:48.834792 18489 leader_election.cc:290] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:48.835000 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:48.835861 18245 raft_consensus.cc:359] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:17:48.836355 17941 leader_election.cc:343] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.836563 18245 raft_consensus.cc:385] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.837159 18245 raft_consensus.cc:740] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.835006 17873 catalog_manager.cc:5697] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:48.837879 18246 raft_consensus.cc:359] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.838629 18246 raft_consensus.cc:385] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.838912 18246 raft_consensus.cc:740] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.836472 18247 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:48.839799 18247 ts_tablet_manager.cc:1403] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.189s	user 0.024s	sys 0.022s
I20260709 13:17:48.838172 18245 consensus_queue.cc:260] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.842494 18247 raft_consensus.cc:359] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.843134 18247 raft_consensus.cc:385] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.827108 18244 raft_consensus.cc:385] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.843384 18247 raft_consensus.cc:740] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.843607 18245 ts_tablet_manager.cc:1434] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:17:48.843532 18244 raft_consensus.cc:740] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.844733 18245 tablet_bootstrap.cc:492] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:48.844947 18244 consensus_queue.cc:260] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.837129 17941 leader_election.cc:304] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:48.850502 18246 consensus_queue.cc:260] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.853431 18246 ts_tablet_manager.cc:1434] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:17:48.854319 18246 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:48.855638 18489 raft_consensus.cc:2749] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:48.855892 18244 ts_tablet_manager.cc:1434] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 13:17:48.856879 18244 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:48.851228 18247 consensus_queue.cc:260] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.865530 18247 ts_tablet_manager.cc:1434] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.003s
I20260709 13:17:48.867486 18247 tablet_bootstrap.cc:492] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:48.850900 18245 tablet_bootstrap.cc:654] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.869169 18453 raft_consensus.cc:493] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.871796 18453 raft_consensus.cc:515] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.873809 18453 leader_election.cc:290] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:48.875310 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:48.876888 18090 leader_election.cc:343] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.878301 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:48.879879 18247 tablet_bootstrap.cc:654] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:48.880752 18089 leader_election.cc:343] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.881253 18246 tablet_bootstrap.cc:654] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.884840 18244 tablet_bootstrap.cc:654] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.890000 18489 consensus_queue.cc:1048] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:48.881217 18089 leader_election.cc:304] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:48.896020 18453 raft_consensus.cc:2749] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:48.902168 18453 raft_consensus.cc:493] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.902732 18453 raft_consensus.cc:515] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:48.903240 18489 consensus_queue.cc:1048] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:48.905100 18453 leader_election.cc:290] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:48.908301 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:48.909185 18198 raft_consensus.cc:2393] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 1: Already voted for candidate 242d2387755e4a37ade4260e80f4eb6a in this term.
I20260709 13:17:48.909330 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:48.911690 18089 leader_election.cc:304] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:48.913069 18453 raft_consensus.cc:3060] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:48.928468 18247 tablet_bootstrap.cc:492] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:48.930330 18247 ts_tablet_manager.cc:1403] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.064s	user 0.005s	sys 0.010s
I20260709 13:17:48.935842 18247 raft_consensus.cc:359] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.953080 18247 raft_consensus.cc:385] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:48.953521 18247 raft_consensus.cc:740] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:48.954270 18247 consensus_queue.cc:260] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.948001 18460 raft_consensus.cc:493] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.957510 18460 raft_consensus.cc:515] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:48.958720 18247 ts_tablet_manager.cc:1434] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:17:48.960832 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:48.962008 18049 raft_consensus.cc:2468] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:48.962450 18247 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:48.964052 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:48.967682 18164 leader_election.cc:304] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:48.968742 18419 raft_consensus.cc:2804] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:48.969131 18419 raft_consensus.cc:493] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.969444 18419 raft_consensus.cc:3060] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:48.971841 18166 leader_election.cc:343] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.973675 18453 raft_consensus.cc:2749] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:48.980063 18247 tablet_bootstrap.cc:654] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:48.984999 18345 raft_consensus.cc:493] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.985564 18345 raft_consensus.cc:515] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:48.986069 18447 raft_consensus.cc:493] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.986546 18447 raft_consensus.cc:515] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:48.988687 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:48.989329 18447 leader_election.cc:290] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:48.990928 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:48.993407 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:48.993868 18049 raft_consensus.cc:2468] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
W20260709 13:17:48.995080 17943 leader_election.cc:343] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:48.992214 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:48.991765 18345 leader_election.cc:290] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:48.996793 18198 raft_consensus.cc:2468] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:48.997949 17940 leader_election.cc:304] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:48.998710 18345 raft_consensus.cc:2804] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:48.999120 18345 raft_consensus.cc:493] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.999526 18345 raft_consensus.cc:3060] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.000136 18498 raft_consensus.cc:493] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.000593 18498 raft_consensus.cc:515] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.002585 18498 leader_election.cc:290] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:49.006372 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.007978 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.008814 17975 raft_consensus.cc:2393] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:49.010546 18017 leader_election.cc:304] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:49.011893 18498 raft_consensus.cc:3060] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:49.012481 17941 leader_election.cc:343] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.013073 17941 leader_election.cc:304] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:49.013893 18252 raft_consensus.cc:2749] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.020829 18419 raft_consensus.cc:515] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.029600 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
W20260709 13:17:49.031430 18166 leader_election.cc:343] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.033329 18460 leader_election.cc:290] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:49.045636 18244 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:49.046164 18244 ts_tablet_manager.cc:1403] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.189s	user 0.008s	sys 0.012s
I20260709 13:17:49.048897 18244 raft_consensus.cc:359] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.049551 18244 raft_consensus.cc:385] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.049829 18244 raft_consensus.cc:740] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.051016 18244 consensus_queue.cc:260] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.052250 18453 raft_consensus.cc:493] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.053010 18453 raft_consensus.cc:515] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.053433 18244 ts_tablet_manager.cc:1434] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:49.054386 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:49.057569 18050 raft_consensus.cc:3060] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.056931 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.065686 17976 raft_consensus.cc:3060] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.034099 18345 raft_consensus.cc:515] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
W20260709 13:17:49.066650 18091 leader_election.cc:343] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.060473 18498 raft_consensus.cc:2749] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.062476 18453 leader_election.cc:290] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:49.054678 18244 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:49.066664 18419 leader_election.cc:290] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:49.075908 18345 leader_election.cc:290] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:49.080466 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:49.081115 18048 raft_consensus.cc:3060] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.083585 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:49.084782 18200 raft_consensus.cc:3060] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.089716 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:49.104951 18089 leader_election.cc:343] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.105448 18089 leader_election.cc:304] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:49.106302 18453 raft_consensus.cc:2749] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.114451 18200 raft_consensus.cc:2468] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:49.118238 18244 tablet_bootstrap.cc:654] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.119794 18246 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:49.120303 18246 ts_tablet_manager.cc:1403] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.266s	user 0.085s	sys 0.097s
I20260709 13:17:49.114492 18048 raft_consensus.cc:2468] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:49.121209 18489 raft_consensus.cc:493] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.123963 18489 raft_consensus.cc:515] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.119799 17940 leader_election.cc:304] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.130892 18489 leader_election.cc:290] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:49.137706 18050 raft_consensus.cc:2468] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:49.131107 18447 raft_consensus.cc:2804] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.140961 18164 leader_election.cc:304] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:49.143478 18245 tablet_bootstrap.cc:492] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:49.143561 18247 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:49.144090 18245 ts_tablet_manager.cc:1403] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.300s	user 0.015s	sys 0.019s
I20260709 13:17:49.144435 18247 ts_tablet_manager.cc:1403] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.182s	user 0.013s	sys 0.008s
I20260709 13:17:49.143573 18447 raft_consensus.cc:697] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:49.143787 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.148923 18447 consensus_queue.cc:237] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.151614 18245 raft_consensus.cc:359] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:49.152324 17943 leader_election.cc:343] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.152351 18245 raft_consensus.cc:385] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.153614 18245 raft_consensus.cc:740] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.154215 18245 consensus_queue.cc:260] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.154817 18247 raft_consensus.cc:359] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.155584 18247 raft_consensus.cc:385] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.155854 18247 raft_consensus.cc:740] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.156584 18247 consensus_queue.cc:260] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.158542 18245 ts_tablet_manager.cc:1434] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:17:49.158742 18247 ts_tablet_manager.cc:1434] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:17:49.159685 18247 tablet_bootstrap.cc:492] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:49.144222 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:49.163231 18419 raft_consensus.cc:493] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.163841 18419 raft_consensus.cc:515] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.148684 18538 raft_consensus.cc:2804] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.166656 18419 leader_election.cc:290] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:49.173492 17940 leader_election.cc:343] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.184616 18245 tablet_bootstrap.cc:492] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:49.181530 18538 raft_consensus.cc:697] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:49.188179 18538 consensus_queue.cc:237] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.180752 17940 leader_election.cc:304] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:49.190871 18489 raft_consensus.cc:2749] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.199440 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.200120 17977 raft_consensus.cc:2468] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:49.203779 18246 raft_consensus.cc:359] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.204469 18246 raft_consensus.cc:385] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.204710 18246 raft_consensus.cc:740] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.205819 17873 catalog_manager.cc:5697] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.207441 18247 tablet_bootstrap.cc:654] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.207707 18165 leader_election.cc:304] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.208537 18419 raft_consensus.cc:2804] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.208943 18419 raft_consensus.cc:493] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.209281 18419 raft_consensus.cc:3060] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.211261 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:49.216476 18246 consensus_queue.cc:260] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
W20260709 13:17:49.219425 18164 leader_election.cc:343] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.220638 18536 raft_consensus.cc:493] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.221130 18536 raft_consensus.cc:515] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.223205 18536 leader_election.cc:290] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:49.224684 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:49.226468 18164 leader_election.cc:343] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.225658 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:49.228341 18165 leader_election.cc:343] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.228789 18165 leader_election.cc:304] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:49.229566 18535 raft_consensus.cc:2749] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.240464 18500 consensus_queue.cc:1048] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.244956 18246 ts_tablet_manager.cc:1434] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.124s	user 0.004s	sys 0.002s
I20260709 13:17:49.245816 18246 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:49.246840 18245 tablet_bootstrap.cc:654] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.249573 18500 raft_consensus.cc:493] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.250139 18500 raft_consensus.cc:515] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.250763 18246 tablet_bootstrap.cc:654] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.251331 18419 raft_consensus.cc:515] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.252882 18500 leader_election.cc:290] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:49.254176 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:49.252997 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:49.255905 18048 raft_consensus.cc:2393] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:49.254124 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:49.260315 17977 raft_consensus.cc:3060] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.266271 18090 leader_election.cc:304] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:49.286553 18500 raft_consensus.cc:3060] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.291883 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:49.293815 18319 raft_consensus.cc:493] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.294358 18319 raft_consensus.cc:515] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.296425 18500 raft_consensus.cc:2749] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.298506 17875 catalog_manager.cc:5697] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:49.300388 18164 leader_election.cc:343] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.304471 18319 leader_election.cc:290] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:49.274227 18419 leader_election.cc:290] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:49.310636 18244 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:49.311188 18244 ts_tablet_manager.cc:1403] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.257s	user 0.037s	sys 0.053s
I20260709 13:17:49.311455 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.314002 17976 raft_consensus.cc:2393] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:49.314347 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.317312 18018 leader_election.cc:304] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:49.318387 18345 consensus_queue.cc:1048] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:49.320410 18498 raft_consensus.cc:3060] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.331066 18345 consensus_queue.cc:1048] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.338656 17977 raft_consensus.cc:2468] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:49.339834 18247 tablet_bootstrap.cc:492] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:49.340416 18247 ts_tablet_manager.cc:1403] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.181s	user 0.013s	sys 0.009s
I20260709 13:17:49.343114 18498 raft_consensus.cc:2749] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.345510 18165 leader_election.cc:304] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:49.346671 18419 raft_consensus.cc:2804] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.347164 18419 raft_consensus.cc:697] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:49.347994 18419 consensus_queue.cc:237] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.348742 18244 raft_consensus.cc:359] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.346499 18293 consensus_queue.cc:1048] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.349421 18244 raft_consensus.cc:385] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.349813 18244 raft_consensus.cc:740] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.350585 18293 consensus_queue.cc:1048] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.373456 18500 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc8a9ec31a5745d69dae4262b271e568)
I20260709 13:17:49.374035 18500 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.374825 18498 raft_consensus.cc:493] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.375346 18498 raft_consensus.cc:515] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.379753 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285967e1504c483dac972f54a3d59bfc" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:49.380999 18018 leader_election.cc:343] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.381044 18293 consensus_queue.cc:1048] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:49.381924 18244 consensus_queue.cc:260] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.383394 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:49.384157 18197 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:17:49.385125 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285967e1504c483dac972f54a3d59bfc" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:49.386376 18015 leader_election.cc:343] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.385185 18089 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.387019 18015 leader_election.cc:304] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:49.387799 18293 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:49.388163 18293 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fc8a9ec31a5745d69dae4262b271e568)
I20260709 13:17:49.388007 18319 raft_consensus.cc:2749] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.389405 18498 leader_election.cc:290] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:49.388587 18293 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:49.391741 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:49.390566 18246 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:49.395854 18246 ts_tablet_manager.cc:1403] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.150s	user 0.012s	sys 0.012s
I20260709 13:17:49.390595 18247 raft_consensus.cc:359] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.400519 18247 raft_consensus.cc:385] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.400897 18247 raft_consensus.cc:740] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.401095 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.401935 17975 raft_consensus.cc:2393] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 2: Already voted for candidate 242d2387755e4a37ade4260e80f4eb6a in this term.
I20260709 13:17:49.401546 18247 consensus_queue.cc:260] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.404101 18244 ts_tablet_manager.cc:1434] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.092s	user 0.006s	sys 0.000s
I20260709 13:17:49.405050 18244 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:49.406189 18293 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.399873 18500 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:49.410663 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:49.423355 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:49.424410 18245 tablet_bootstrap.cc:492] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:49.425104 18245 ts_tablet_manager.cc:1403] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.241s	user 0.016s	sys 0.002s
I20260709 13:17:49.428740 18293 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:49.429515 17874 catalog_manager.cc:5697] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.430670 18197 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:17:49.431711 18089 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.433194 18453 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:49.433656 18453 raft_consensus.cc:697] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:49.434401 18453 consensus_queue.cc:237] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.424165 17976 raft_consensus.cc:2393] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 2: Already voted for candidate 242d2387755e4a37ade4260e80f4eb6a in this term.
I20260709 13:17:49.452666 18244 tablet_bootstrap.cc:654] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.463989 18244 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:49.464433 18244 ts_tablet_manager.cc:1403] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.002s
W20260709 13:17:49.467383 18551 log.cc:927] Time spent T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:17:49.479496 18246 raft_consensus.cc:359] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.469643 18245 raft_consensus.cc:359] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.505859 18245 raft_consensus.cc:385] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.506197 18245 raft_consensus.cc:740] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.472755 18244 raft_consensus.cc:359] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.507706 18244 raft_consensus.cc:385] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.507992 18244 raft_consensus.cc:740] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.508627 18244 consensus_queue.cc:260] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.485313 18447 raft_consensus.cc:493] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.510351 18447 raft_consensus.cc:515] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.506863 18245 consensus_queue.cc:260] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.512490 18447 leader_election.cc:290] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:49.513566 18245 ts_tablet_manager.cc:1434] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.088s	user 0.003s	sys 0.007s
I20260709 13:17:49.514088 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.514590 18245 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:49.515153 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:49.478966 17874 catalog_manager.cc:5697] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 1 to 2, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:49.515307 17943 leader_election.cc:343] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:49.520529 17941 leader_election.cc:343] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.521405 17941 leader_election.cc:304] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:49.522185 18447 raft_consensus.cc:2749] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:49.522341 18246 raft_consensus.cc:385] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.522746 18246 raft_consensus.cc:740] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.523355 18246 consensus_queue.cc:260] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.525519 18245 tablet_bootstrap.cc:654] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.528759 18246 ts_tablet_manager.cc:1434] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.133s	user 0.010s	sys 0.003s
I20260709 13:17:49.529649 18246 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:49.489087 18247 ts_tablet_manager.cc:1434] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.148s	user 0.004s	sys 0.002s
I20260709 13:17:49.530645 18247 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:49.516963 18244 ts_tablet_manager.cc:1434] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.052s	user 0.004s	sys 0.005s
I20260709 13:17:49.534063 18244 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:49.536070 18246 tablet_bootstrap.cc:654] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.533416 18245 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:49.538509 18245 ts_tablet_manager.cc:1403] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 13:17:49.541671 18245 raft_consensus.cc:359] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.542871 18245 raft_consensus.cc:385] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.543128 18245 raft_consensus.cc:740] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.543962 18245 consensus_queue.cc:260] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.546136 18245 ts_tablet_manager.cc:1434] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:49.547149 18245 tablet_bootstrap.cc:492] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:49.552899 18244 tablet_bootstrap.cc:654] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.554034 18245 tablet_bootstrap.cc:654] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.561638 18247 tablet_bootstrap.cc:654] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.565076 18447 consensus_queue.cc:1048] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:17:49.576267 18246 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:49.581658 18246 ts_tablet_manager.cc:1403] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.019s
I20260709 13:17:49.594543 18447 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.595172 18244 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:49.595752 18244 ts_tablet_manager.cc:1403] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.018s
I20260709 13:17:49.595139 18447 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.605028 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.605131 18447 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:49.605351 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:49.605980 18200 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:17:49.607234 17940 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:49.614375 18447 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.614768 18447 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.615105 18447 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.621963 18447 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.624794 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:49.626045 18246 raft_consensus.cc:359] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.626724 18447 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:49.626822 18246 raft_consensus.cc:385] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.629747 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:49.630921 18245 tablet_bootstrap.cc:492] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:49.636034 18246 raft_consensus.cc:740] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.636303 18247 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:49.636473 18245 ts_tablet_manager.cc:1403] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.090s	user 0.013s	sys 0.017s
I20260709 13:17:49.636773 18247 ts_tablet_manager.cc:1403] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.106s	user 0.022s	sys 0.017s
I20260709 13:17:49.636734 18246 consensus_queue.cc:260] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.636343 18198 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.639024 18246 ts_tablet_manager.cc:1434] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.057s	user 0.002s	sys 0.002s
I20260709 13:17:49.639336 18245 raft_consensus.cc:359] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.639672 18247 raft_consensus.cc:359] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.640084 18245 raft_consensus.cc:385] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.640380 18247 raft_consensus.cc:385] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.640763 18247 raft_consensus.cc:740] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.641412 18247 consensus_queue.cc:260] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.647254 18247 ts_tablet_manager.cc:1434] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:17:49.648844 18247 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:49.639998 18246 tablet_bootstrap.cc:492] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:49.640470 18245 raft_consensus.cc:740] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.651093 18245 consensus_queue.cc:260] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.651794 18498 raft_consensus.cc:493] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.652423 18498 raft_consensus.cc:515] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.654757 18498 leader_election.cc:290] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:49.656102 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:49.656494 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.666181 18126 raft_consensus.cc:2468] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:49.655934 18246 tablet_bootstrap.cc:654] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:49.667951 18015 leader_election.cc:343] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.663699 18245 ts_tablet_manager.cc:1434] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.003s
I20260709 13:17:49.669942 18018 leader_election.cc:304] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:49.670528 18245 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:49.652308 18447 consensus_queue.cc:1048] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:17:49.674731 18319 raft_consensus.cc:2804] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.675168 18319 raft_consensus.cc:493] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.675510 18319 raft_consensus.cc:3060] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.684137 18319 raft_consensus.cc:515] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.689759 18245 tablet_bootstrap.cc:654] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.691912 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:49.693979 18126 raft_consensus.cc:3060] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.635210 18244 raft_consensus.cc:359] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.694486 18244 raft_consensus.cc:385] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.701045 18244 raft_consensus.cc:740] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.701712 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
W20260709 13:17:49.703190 18015 leader_election.cc:343] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.700572 18319 leader_election.cc:290] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:49.685175 18247 tablet_bootstrap.cc:654] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.704986 18126 raft_consensus.cc:2468] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:49.701896 18244 consensus_queue.cc:260] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.690488 18198 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 3.
I20260709 13:17:49.709484 18246 tablet_bootstrap.cc:492] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:49.710024 18246 ts_tablet_manager.cc:1403] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.001s
I20260709 13:17:49.710808 17940 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230
I20260709 13:17:49.721028 18489 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:49.721733 18489 raft_consensus.cc:697] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:49.722791 18489 consensus_queue.cc:237] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.710503 18018 leader_election.cc:304] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:49.712294 18244 ts_tablet_manager.cc:1434] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.116s	user 0.006s	sys 0.000s
I20260709 13:17:49.724550 18246 raft_consensus.cc:359] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.725134 18319 raft_consensus.cc:2804] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.730508 18245 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:49.730235 18319 raft_consensus.cc:697] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:49.731875 18319 consensus_queue.cc:237] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.725224 18244 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:49.725279 18246 raft_consensus.cc:385] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.736028 18246 raft_consensus.cc:740] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.736896 18246 consensus_queue.cc:260] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.731055 18245 ts_tablet_manager.cc:1403] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.000s
I20260709 13:17:49.727815 18538 consensus_queue.cc:1048] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:17:49.745811 18489 raft_consensus.cc:493] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.746006 18535 consensus_queue.cc:1048] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:49.746429 18489 raft_consensus.cc:515] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.745677 18244 tablet_bootstrap.cc:654] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.747469 18535 consensus_queue.cc:1048] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:17:49.751845 18245 raft_consensus.cc:359] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.752502 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:49.752537 18245 raft_consensus.cc:385] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.760113 18245 raft_consensus.cc:740] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.760869 18245 consensus_queue.cc:260] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.763842 18200 raft_consensus.cc:2468] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:49.767993 18555 raft_consensus.cc:493] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.762519 18554 raft_consensus.cc:493] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.769307 18555 raft_consensus.cc:515] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.769948 18554 raft_consensus.cc:515] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.771977 18554 leader_election.cc:290] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:49.772586 17940 leader_election.cc:304] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.773453 18345 raft_consensus.cc:2804] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.773854 18345 raft_consensus.cc:493] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.774176 18345 raft_consensus.cc:3060] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.776921 17874 catalog_manager.cc:5697] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 2 to 3, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 3 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:49.780472 18245 ts_tablet_manager.cc:1434] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:17:49.784322 18246 ts_tablet_manager.cc:1434] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.074s	user 0.009s	sys 0.005s
I20260709 13:17:49.786940 18245 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:49.788638 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:49.789391 18200 raft_consensus.cc:2468] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:49.779455 18555 leader_election.cc:290] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:49.804303 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.804533 18345 raft_consensus.cc:515] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.805147 17976 raft_consensus.cc:2468] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:49.806890 18247 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:49.807435 18247 ts_tablet_manager.cc:1403] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.159s	user 0.023s	sys 0.008s
I20260709 13:17:49.807588 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:49.808161 18200 raft_consensus.cc:3060] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.785658 18489 leader_election.cc:290] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:49.810254 18089 leader_election.cc:304] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.811125 18555 raft_consensus.cc:2804] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.811496 18555 raft_consensus.cc:493] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.811830 18555 raft_consensus.cc:3060] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.812288 18345 leader_election.cc:290] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:49.816803 18246 tablet_bootstrap.cc:492] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:49.822288 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:49.822192 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:49.832397 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.833276 17976 raft_consensus.cc:2468] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
W20260709 13:17:49.855381 17941 leader_election.cc:343] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.855862 18091 leader_election.cc:304] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:49.856966 18453 raft_consensus.cc:2804] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.857379 18453 raft_consensus.cc:493] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.857703 18453 raft_consensus.cc:3060] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.848594 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:49.860486 18050 raft_consensus.cc:3060] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.861182 18247 raft_consensus.cc:359] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.862001 18247 raft_consensus.cc:385] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.862259 18247 raft_consensus.cc:740] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
W20260709 13:17:49.868629 18565 log.cc:927] Time spent T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272: Append to log took a long time: real 0.066s	user 0.000s	sys 0.002s
I20260709 13:17:49.869081 18419 raft_consensus.cc:493] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.869637 18419 raft_consensus.cc:515] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:49.879317 18535 consensus_queue.cc:1048] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:49.883038 18245 tablet_bootstrap.cc:654] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.881347 18453 raft_consensus.cc:515] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.842404 18555 raft_consensus.cc:515] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.886757 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:49.887427 18198 raft_consensus.cc:3060] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.887686 18555 leader_election.cc:290] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:49.888142 18419 leader_election.cc:290] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:49.889730 18247 consensus_queue.cc:260] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.891695 18052 raft_consensus.cc:2468] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:49.896672 18198 raft_consensus.cc:2468] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:49.897536 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.899420 18124 raft_consensus.cc:2393] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc8a9ec31a5745d69dae4262b271e568 in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:49.900209 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:49.898094 17941 leader_election.cc:343] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:49.898442 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:49.896850 17977 raft_consensus.cc:1240] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Rejecting Update request from peer 1787628aa7f6489497bb871588797230 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.901407 17975 raft_consensus.cc:3060] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.912477 18536 raft_consensus.cc:493] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.913049 18536 raft_consensus.cc:515] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.913954 18089 leader_election.cc:304] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.914505 18200 raft_consensus.cc:2468] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:49.915032 18536 leader_election.cc:290] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:49.913690 17975 raft_consensus.cc:2468] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:49.916476 18164 leader_election.cc:304] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:49.916744 17940 leader_election.cc:304] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:49.918322 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.922916 18555 raft_consensus.cc:2804] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.923329 18247 ts_tablet_manager.cc:1434] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.099s	user 0.005s	sys 0.002s
I20260709 13:17:49.930786 18247 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:49.933928 18345 raft_consensus.cc:2804] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:49.934568 18554 consensus_queue.cc:1059] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:49.932000 18124 raft_consensus.cc:2468] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:49.935285 18554 consensus_queue.cc:1243] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [LEADER]: Peer 242d2387755e4a37ade4260e80f4eb6a log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 13:17:49.935844 18554 raft_consensus.cc:3055] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.936203 18554 raft_consensus.cc:740] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:49.936964 18554 consensus_queue.cc:260] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.938086 18554 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.940282 18166 leader_election.cc:304] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:49.942373 18536 raft_consensus.cc:2804] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:49.942873 18536 raft_consensus.cc:493] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.943138 18246 tablet_bootstrap.cc:654] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:49.943208 18536 raft_consensus.cc:3060] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.944927 18244 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:49.945441 18244 ts_tablet_manager.cc:1403] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.220s	user 0.015s	sys 0.005s
I20260709 13:17:49.947556 18345 raft_consensus.cc:697] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:49.949939 18345 consensus_queue.cc:237] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:49.952106 17873 catalog_manager.cc:5697] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.932783 18571 raft_consensus.cc:3060] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.942247 18447 raft_consensus.cc:493] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.956363 18447 raft_consensus.cc:515] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:49.958123 18244 raft_consensus.cc:359] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.958683 18244 raft_consensus.cc:385] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:49.958879 18244 raft_consensus.cc:740] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:49.932515 18555 raft_consensus.cc:697] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:49.966293 18244 consensus_queue.cc:260] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:49.976449 18555 consensus_queue.cc:237] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:17:49.977640 18573 log.cc:927] Time spent T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:17:49.979159 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:49.983042 18245 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:49.983520 18245 ts_tablet_manager.cc:1403] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.197s	user 0.013s	sys 0.001s
I20260709 13:17:49.982059 18126 raft_consensus.cc:2468] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:49.989499 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:49.990209 17977 raft_consensus.cc:2468] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:49.969250 18453 leader_election.cc:290] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:49.996794 18244 ts_tablet_manager.cc:1434] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 13:17:49.971107 18124 raft_consensus.cc:3060] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:49.999310 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:50.000000 17975 raft_consensus.cc:3060] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.001497 18245 raft_consensus.cc:359] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.008970 18245 raft_consensus.cc:385] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.009235 18245 raft_consensus.cc:740] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.009577 18246 tablet_bootstrap.cc:492] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:50.010058 18246 ts_tablet_manager.cc:1403] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.214s	user 0.017s	sys 0.013s
I20260709 13:17:50.009831 18245 consensus_queue.cc:260] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.021494 18244 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:50.028028 18247 tablet_bootstrap.cc:654] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.036959 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:50.037732 18052 raft_consensus.cc:3060] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.018180 18453 raft_consensus.cc:493] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.025554 17975 raft_consensus.cc:2468] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:50.043859 18447 leader_election.cc:290] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:50.019101 18536 raft_consensus.cc:515] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:50.055174 18246 raft_consensus.cc:359] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.056528 18246 raft_consensus.cc:385] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.059813 18245 ts_tablet_manager.cc:1434] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.076s	user 0.009s	sys 0.004s
I20260709 13:17:50.056725 18246 raft_consensus.cc:740] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.060755 18245 tablet_bootstrap.cc:492] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:50.060739 18246 consensus_queue.cc:260] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.065397 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:50.066187 18198 raft_consensus.cc:2393] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:50.052142 18453 raft_consensus.cc:515] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.076851 18246 ts_tablet_manager.cc:1434] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:17:50.078833 18536 leader_election.cc:290] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:50.080931 18052 raft_consensus.cc:2468] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:50.075510 18571 raft_consensus.cc:2749] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.089713 18090 leader_election.cc:304] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:50.090544 18581 raft_consensus.cc:2804] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:50.091128 18244 tablet_bootstrap.cc:654] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.091059 18581 raft_consensus.cc:697] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:50.098842 17873 catalog_manager.cc:5697] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.100368 18453 leader_election.cc:290] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:50.105532 18570 raft_consensus.cc:493] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.106022 18570 raft_consensus.cc:515] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.108598 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:50.110066 18164 leader_election.cc:343] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.111102 18570 leader_election.cc:290] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:50.111321 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:50.111994 17975 raft_consensus.cc:3060] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.113735 17943 leader_election.cc:304] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:50.114464 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:50.115774 18090 leader_election.cc:343] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.118719 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:50.119351 18198 raft_consensus.cc:2468] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:50.121035 18581 consensus_queue.cc:237] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.122208 17975 raft_consensus.cc:2468] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:50.123152 18089 leader_election.cc:304] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:50.124712 18246 tablet_bootstrap.cc:492] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:50.127763 18245 tablet_bootstrap.cc:654] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.132387 18252 raft_consensus.cc:2764] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:17:50.136792 18165 leader_election.cc:304] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:50.138343 18570 raft_consensus.cc:2804] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:50.141323 18570 raft_consensus.cc:697] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:50.137468 18555 raft_consensus.cc:2804] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:50.144330 18555 raft_consensus.cc:493] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.144786 18555 raft_consensus.cc:3060] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.157136 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7077b3a2710489282f1c1cd3f8c1ddb" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:50.157847 18122 raft_consensus.cc:3060] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.164295 18246 tablet_bootstrap.cc:654] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.167887 17875 catalog_manager.cc:5697] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.142235 18570 consensus_queue.cc:237] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:50.163213 18555 raft_consensus.cc:515] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.175732 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:50.176335 18198 raft_consensus.cc:3060] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.179867 18555 leader_election.cc:290] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:50.179611 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:50.180817 18126 raft_consensus.cc:2393] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fc8a9ec31a5745d69dae4262b271e568 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:50.203508 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
W20260709 13:17:50.205061 18090 leader_election.cc:343] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.204169 18555 raft_consensus.cc:493] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.207558 18555 raft_consensus.cc:515] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.209945 18555 leader_election.cc:290] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:50.211236 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:50.212078 18166 leader_election.cc:304] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
W20260709 13:17:50.213155 18090 leader_election.cc:343] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.215158 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:50.215642 18197 raft_consensus.cc:1275] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:50.216045 17975 raft_consensus.cc:2468] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:50.217335 18489 consensus_queue.cc:1048] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:17:50.225638 18091 leader_election.cc:304] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:50.219614 18122 raft_consensus.cc:2468] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:50.233078 18555 raft_consensus.cc:2804] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:50.234632 18555 raft_consensus.cc:493] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.235047 18555 raft_consensus.cc:3060] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.236588 18245 tablet_bootstrap.cc:492] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:50.237107 18245 ts_tablet_manager.cc:1403] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.177s	user 0.012s	sys 0.011s
I20260709 13:17:50.237277 18247 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:50.237411 18246 tablet_bootstrap.cc:492] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:50.237772 18247 ts_tablet_manager.cc:1403] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.307s	user 0.017s	sys 0.000s
I20260709 13:17:50.237953 18246 ts_tablet_manager.cc:1403] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.003s
I20260709 13:17:50.240897 18246 raft_consensus.cc:359] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.242020 18246 raft_consensus.cc:385] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.244343 18498 consensus_queue.cc:1048] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:50.248452 18125 raft_consensus.cc:1275] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:17:50.248975 18125 pending_rounds.cc:85] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: Aborting all ops after (but not including) 1
I20260709 13:17:50.249236 18125 pending_rounds.cc:107] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:17:50.249527 18125 raft_consensus.cc:2889] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:50.254706 18245 raft_consensus.cc:359] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.255565 18245 raft_consensus.cc:385] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.255955 18245 raft_consensus.cc:740] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.257328 18245 consensus_queue.cc:260] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.264137 18245 ts_tablet_manager.cc:1434] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:17:50.265261 18245 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:50.267280 18252 raft_consensus.cc:493] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.267975 18252 raft_consensus.cc:515] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.269521 18247 raft_consensus.cc:359] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:50.270272 18247 raft_consensus.cc:385] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.270570 18247 raft_consensus.cc:740] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.271260 18247 consensus_queue.cc:260] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:50.273036 18447 consensus_queue.cc:1048] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.273922 18245 tablet_bootstrap.cc:654] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.277179 18247 ts_tablet_manager.cc:1434] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:17:50.281421 18247 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:50.281919 18555 raft_consensus.cc:515] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.282817 18244 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:50.283360 18244 ts_tablet_manager.cc:1403] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.262s	user 0.016s	sys 0.000s
I20260709 13:17:50.285099 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:50.242324 18246 raft_consensus.cc:740] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.285725 17975 raft_consensus.cc:3060] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.286118 18244 raft_consensus.cc:359] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:50.286813 18244 raft_consensus.cc:385] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.287647 18244 raft_consensus.cc:740] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.286813 18246 consensus_queue.cc:260] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.255221 18198 raft_consensus.cc:2468] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:50.289541 18536 raft_consensus.cc:2749] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.270100 18252 leader_election.cc:290] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:50.292490 18089 leader_election.cc:304] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:50.293977 18583 raft_consensus.cc:2804] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:50.294641 18583 raft_consensus.cc:697] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:50.275835 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:50.297923 17941 leader_election.cc:343] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.299233 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:50.300413 18557 raft_consensus.cc:493] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fc8a9ec31a5745d69dae4262b271e568)
I20260709 13:17:50.300980 18557 raft_consensus.cc:515] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:50.301321 17943 leader_election.cc:343] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.301820 17943 leader_election.cc:304] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:50.306872 18447 raft_consensus.cc:2749] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.312633 17875 catalog_manager.cc:5697] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.323706 18555 leader_election.cc:290] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:50.311223 18244 consensus_queue.cc:260] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:50.332928 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:50.336447 18358 consensus_queue.cc:1048] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:17:50.337450 18090 leader_election.cc:343] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.334429 18583 consensus_queue.cc:237] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.340186 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:50.338696 18557 leader_election.cc:290] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:50.341672 17873 catalog_manager.cc:5697] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.351752 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:50.356895 18246 ts_tablet_manager.cc:1434] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.119s	user 0.007s	sys 0.002s
I20260709 13:17:50.358206 18246 tablet_bootstrap.cc:492] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:50.369359 18090 leader_election.cc:304] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:50.370388 18453 raft_consensus.cc:2749] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:50.381945 18244 ts_tablet_manager.cc:1434] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.000s
I20260709 13:17:50.383315 18244 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:50.400384 18245 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:50.400949 18245 ts_tablet_manager.cc:1403] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.136s	user 0.019s	sys 0.017s
I20260709 13:17:50.403919 18245 raft_consensus.cc:359] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.404568 18245 raft_consensus.cc:385] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.404827 18245 raft_consensus.cc:740] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.405548 18245 consensus_queue.cc:260] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.410964 18244 tablet_bootstrap.cc:654] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.418577 18245 ts_tablet_manager.cc:1434] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:17:50.419621 18245 tablet_bootstrap.cc:492] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:50.428397 18245 tablet_bootstrap.cc:654] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.434155 17975 raft_consensus.cc:2468] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:50.435750 18091 leader_election.cc:304] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:50.436702 18555 raft_consensus.cc:2804] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:50.437213 18555 raft_consensus.cc:697] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:50.438045 18555 consensus_queue.cc:237] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.440263 18557 raft_consensus.cc:493] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.440793 18557 raft_consensus.cc:515] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.444103 18557 leader_election.cc:290] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:50.444959 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:50.446067 17975 raft_consensus.cc:2468] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:50.447221 18091 leader_election.cc:304] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:50.448021 18247 tablet_bootstrap.cc:654] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.448417 18557 raft_consensus.cc:2804] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:50.445133 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:50.448916 18557 raft_consensus.cc:493] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.449623 18557 raft_consensus.cc:3060] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:50.450188 18090 leader_election.cc:343] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.454109 18246 tablet_bootstrap.cc:654] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.462430 18557 raft_consensus.cc:515] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.470048 18246 tablet_bootstrap.cc:492] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:50.472504 18246 ts_tablet_manager.cc:1403] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.115s	user 0.010s	sys 0.001s
I20260709 13:17:50.488443 18453 consensus_queue.cc:1048] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:50.503593 18246 raft_consensus.cc:359] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.516072 18246 raft_consensus.cc:385] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.507583 17974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:50.516539 18246 raft_consensus.cc:740] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.513919 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:50.515111 18453 consensus_queue.cc:1048] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.517896 17974 raft_consensus.cc:3060] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.517668 18557 leader_election.cc:290] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:50.518177 18246 consensus_queue.cc:260] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.527906 18247 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:50.528597 18247 ts_tablet_manager.cc:1403] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.247s	user 0.018s	sys 0.004s
I20260709 13:17:50.536918 18051 raft_consensus.cc:3060] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.538929 18245 tablet_bootstrap.cc:492] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:50.539531 18245 ts_tablet_manager.cc:1403] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.120s	user 0.015s	sys 0.012s
I20260709 13:17:50.531668 17872 catalog_manager.cc:5697] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.545315 18247 raft_consensus.cc:359] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
W20260709 13:17:50.545331 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:50.546097 18247 raft_consensus.cc:385] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.547969 18247 raft_consensus.cc:740] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.548584 18247 consensus_queue.cc:260] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.559312 18246 ts_tablet_manager.cc:1434] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.000s
I20260709 13:17:50.552173 18245 raft_consensus.cc:359] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.575917 18245 raft_consensus.cc:385] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.576239 18245 raft_consensus.cc:740] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.555042 18244 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:50.568892 18247 ts_tablet_manager.cc:1434] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 13:17:50.577077 18244 ts_tablet_manager.cc:1403] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.194s	user 0.017s	sys 0.009s
I20260709 13:17:50.577893 18246 tablet_bootstrap.cc:492] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:50.578357 18247 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:50.558555 17974 raft_consensus.cc:2468] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
W20260709 13:17:50.551972 18090 leader_election.cc:343] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.576949 18245 consensus_queue.cc:260] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.591233 18244 raft_consensus.cc:359] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.591990 18244 raft_consensus.cc:385] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.592262 18244 raft_consensus.cc:740] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.596673 18245 ts_tablet_manager.cc:1434] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 13:17:50.598748 18252 consensus_queue.cc:1048] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.606050 18252 consensus_queue.cc:1048] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.608651 18453 consensus_queue.cc:1048] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.610129 18091 leader_election.cc:304] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:50.610684 18245 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:50.597275 18244 consensus_queue.cc:260] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.618353 18247 tablet_bootstrap.cc:654] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.619185 18555 raft_consensus.cc:2804] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:50.621130 17874 catalog_manager.cc:5697] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.621146 18555 raft_consensus.cc:697] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:50.625066 18246 tablet_bootstrap.cc:654] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.624830 18555 consensus_queue.cc:237] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.633153 18557 consensus_queue.cc:1048] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.648933 18453 raft_consensus.cc:493] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.649528 18453 raft_consensus.cc:515] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.652633 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285967e1504c483dac972f54a3d59bfc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:50.653362 18052 raft_consensus.cc:2468] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:50.654726 18090 leader_election.cc:304] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:50.655615 18554 raft_consensus.cc:2804] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:50.655963 18554 raft_consensus.cc:493] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.656282 18554 raft_consensus.cc:3060] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.656672 18244 ts_tablet_manager.cc:1434] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.000s
I20260709 13:17:50.657142 18453 leader_election.cc:290] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:50.659049 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285967e1504c483dac972f54a3d59bfc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:50.659763 18200 raft_consensus.cc:2468] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:50.662834 18244 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:50.670333 18498 raft_consensus.cc:493] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.670877 18498 raft_consensus.cc:515] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.672833 18498 leader_election.cc:290] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:50.674834 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:50.675601 17977 raft_consensus.cc:2393] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:50.675459 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:50.683207 18447 raft_consensus.cc:493] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.684016 18447 raft_consensus.cc:515] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.686749 18018 leader_election.cc:304] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:50.687047 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd55fcdd2224c1db74729d68acc0ee6" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:50.687805 18245 tablet_bootstrap.cc:654] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.688340 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd55fcdd2224c1db74729d68acc0ee6" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:50.688598 17943 leader_election.cc:343] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:50.689787 17941 leader_election.cc:343] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.690205 17941 leader_election.cc:304] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:50.690215 18447 leader_election.cc:290] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:50.697032 17874 catalog_manager.cc:5697] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.702033 18498 raft_consensus.cc:3060] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.698535 18252 raft_consensus.cc:2749] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.698865 18489 raft_consensus.cc:493] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.703266 18489 raft_consensus.cc:515] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:50.705387 18489 leader_election.cc:290] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:50.707196 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:50.709007 17940 leader_election.cc:343] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.679082 18554 raft_consensus.cc:515] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.711869 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:50.713550 17941 leader_election.cc:343] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.713974 17941 leader_election.cc:304] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:50.714982 18489 raft_consensus.cc:2749] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.737573 18244 tablet_bootstrap.cc:654] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.759759 18554 leader_election.cc:290] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:50.761762 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285967e1504c483dac972f54a3d59bfc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:50.762424 18200 raft_consensus.cc:3060] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.772890 18498 raft_consensus.cc:2749] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.773211 18583 raft_consensus.cc:493] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.773694 18583 raft_consensus.cc:515] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.776898 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb77b44e31244ec29dad965bc258249a" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:50.777995 18247 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
W20260709 13:17:50.778258 18089 leader_election.cc:343] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.778476 18247 ts_tablet_manager.cc:1403] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.200s	user 0.009s	sys 0.032s
I20260709 13:17:50.779066 18583 leader_election.cc:290] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:50.779968 18200 raft_consensus.cc:2468] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:50.781277 18089 leader_election.cc:304] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:50.788053 18245 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:50.788589 18245 ts_tablet_manager.cc:1403] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.178s	user 0.037s	sys 0.016s
I20260709 13:17:50.790616 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb77b44e31244ec29dad965bc258249a" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:50.791432 18245 raft_consensus.cc:359] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.792136 18245 raft_consensus.cc:385] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.791666 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "285967e1504c483dac972f54a3d59bfc" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:50.792436 18245 raft_consensus.cc:740] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.793124 18245 consensus_queue.cc:260] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.793972 18583 raft_consensus.cc:2804] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:50.794595 18583 raft_consensus.cc:697] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
W20260709 13:17:50.798435 18090 leader_election.cc:343] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.805446 18090 leader_election.cc:304] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:50.803130 18052 raft_consensus.cc:3060] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:50.806398 18245 ts_tablet_manager.cc:1434] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:17:50.811772 18245 tablet_bootstrap.cc:492] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:50.807557 18583 consensus_queue.cc:237] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.811944 18538 raft_consensus.cc:493] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.812608 18538 raft_consensus.cc:515] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.813040 18453 consensus_queue.cc:1048] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:50.808377 18554 raft_consensus.cc:2749] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:50.808619 18536 consensus_queue.cc:1048] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.809119 18246 tablet_bootstrap.cc:492] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:50.827716 18245 tablet_bootstrap.cc:654] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.839802 18246 ts_tablet_manager.cc:1403] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.263s	user 0.009s	sys 0.009s
I20260709 13:17:50.819828 18247 raft_consensus.cc:359] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:50.857605 18247 raft_consensus.cc:385] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.857921 18247 raft_consensus.cc:740] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
W20260709 13:17:50.861148 18090 consensus_peers.cc:597] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:50.848680 18570 consensus_queue.cc:1048] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:17:50.832010 18052 raft_consensus.cc:2468] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:50.858701 18247 consensus_queue.cc:260] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:50.868079 18538 leader_election.cc:290] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:50.877310 18583 consensus_queue.cc:1048] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:50.880185 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:50.882064 18166 leader_election.cc:343] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:50.881759 18246 raft_consensus.cc:359] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.884642 18246 raft_consensus.cc:385] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.884900 18246 raft_consensus.cc:740] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.887099 18247 ts_tablet_manager.cc:1434] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.108s	user 0.009s	sys 0.000s
I20260709 13:17:50.888170 18247 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:50.901088 18583 consensus_queue.cc:1048] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:50.906764 18246 consensus_queue.cc:260] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.913913 18244 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:50.914551 18244 ts_tablet_manager.cc:1403] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.252s	user 0.034s	sys 0.019s
I20260709 13:17:50.925937 18244 raft_consensus.cc:359] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.927532 18244 raft_consensus.cc:385] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.928372 18244 raft_consensus.cc:740] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
W20260709 13:17:50.930931 18619 log.cc:927] Time spent T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:17:50.939916 18244 consensus_queue.cc:260] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:50.944639 18246 ts_tablet_manager.cc:1434] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.098s	user 0.004s	sys 0.003s
I20260709 13:17:50.945659 18246 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
W20260709 13:17:50.932600 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 85843.5us
I20260709 13:17:50.951639 18244 ts_tablet_manager.cc:1434] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:17:50.952925 18244 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:50.971385 18246 tablet_bootstrap.cc:654] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.984182 18245 tablet_bootstrap.cc:492] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:50.984680 18245 ts_tablet_manager.cc:1403] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.173s	user 0.013s	sys 0.009s
I20260709 13:17:50.986814 18244 tablet_bootstrap.cc:654] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:50.988502 18245 raft_consensus.cc:359] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.989218 18245 raft_consensus.cc:385] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:50.990899 18245 raft_consensus.cc:740] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:50.991648 18245 consensus_queue.cc:260] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:50.998409 18245 ts_tablet_manager.cc:1434] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:17:50.999570 18245 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:50.983356 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:51.051108 18555 consensus_queue.cc:1048] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:51.052824 18245 tablet_bootstrap.cc:654] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.059909 18498 raft_consensus.cc:493] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.060534 18498 raft_consensus.cc:515] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
W20260709 13:17:51.059535 18164 leader_election.cc:343] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.063781 18498 leader_election.cc:290] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:51.073184 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:51.074083 17975 raft_consensus.cc:2468] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:51.075971 18017 leader_election.cc:304] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:51.076804 18498 raft_consensus.cc:2804] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.077183 18498 raft_consensus.cc:493] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.077500 18498 raft_consensus.cc:3060] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.089291 18246 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:51.089797 18246 ts_tablet_manager.cc:1403] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.144s	user 0.027s	sys 0.017s
I20260709 13:17:51.091018 18247 tablet_bootstrap.cc:654] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.091459 18555 raft_consensus.cc:493] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.092374 18246 raft_consensus.cc:359] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.093084 18246 raft_consensus.cc:385] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.092730 18555 raft_consensus.cc:515] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.093361 18246 raft_consensus.cc:740] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.099689 18246 consensus_queue.cc:260] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.061553 18164 leader_election.cc:304] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:51.094187 17873 catalog_manager.cc:5697] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.115291 18620 raft_consensus.cc:2749] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:51.114787 18555 leader_election.cc:290] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:51.121706 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691cfc20fdd444b3b53db6fdcdedcda8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:51.122426 18198 raft_consensus.cc:2468] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:51.123762 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691cfc20fdd444b3b53db6fdcdedcda8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:51.126698 18246 ts_tablet_manager.cc:1434] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:17:51.127767 18246 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:51.134889 18246 tablet_bootstrap.cc:654] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.086026 18498 raft_consensus.cc:515] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.138137 18089 leader_election.cc:304] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:51.139761 18453 raft_consensus.cc:2804] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.146777 18453 raft_consensus.cc:493] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.148132 18453 raft_consensus.cc:3060] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:51.140141 18090 leader_election.cc:343] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:51.142256 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 85031.7us
I20260709 13:17:51.158011 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:51.160748 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:51.161381 17973 raft_consensus.cc:3060] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.164408 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
W20260709 13:17:51.169430 18015 leader_election.cc:343] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.174793 17973 raft_consensus.cc:2468] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:51.177752 18017 leader_election.cc:304] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:51.178365 18244 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:51.168584 18498 leader_election.cc:290] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:51.178860 18244 ts_tablet_manager.cc:1403] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.226s	user 0.019s	sys 0.000s
W20260709 13:17:51.178480 18015 leader_election.cc:343] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.178776 18600 raft_consensus.cc:2804] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:51.180148 18600 raft_consensus.cc:697] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:51.181170 18245 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:51.180974 18600 consensus_queue.cc:237] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.183723 18489 raft_consensus.cc:493] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.184204 18489 raft_consensus.cc:515] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.181640 18245 ts_tablet_manager.cc:1403] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.182s	user 0.010s	sys 0.009s
I20260709 13:17:51.186183 18247 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:51.186688 18247 ts_tablet_manager.cc:1403] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.299s	user 0.034s	sys 0.020s
I20260709 13:17:51.187318 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:51.188709 17943 leader_election.cc:343] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.186151 18489 leader_election.cc:290] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:51.197890 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:51.199039 18453 raft_consensus.cc:515] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.203277 18453 leader_election.cc:290] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:51.204507 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691cfc20fdd444b3b53db6fdcdedcda8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:51.205153 18198 raft_consensus.cc:3060] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:51.208712 17941 leader_election.cc:343] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.209666 17941 leader_election.cc:304] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:51.210397 18622 raft_consensus.cc:2749] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:51.209854 18453 raft_consensus.cc:493] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.211220 18453 raft_consensus.cc:515] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.214630 18247 raft_consensus.cc:359] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.214830 18245 raft_consensus.cc:359] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.215502 18245 raft_consensus.cc:385] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.215804 18245 raft_consensus.cc:740] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.216118 18453 leader_election.cc:290] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:51.216449 18245 consensus_queue.cc:260] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.218152 18247 raft_consensus.cc:385] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.218508 18247 raft_consensus.cc:740] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.219216 18246 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:51.219781 18246 ts_tablet_manager.cc:1403] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.000s
I20260709 13:17:51.220043 18247 consensus_queue.cc:260] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.220422 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691cfc20fdd444b3b53db6fdcdedcda8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:51.227905 18244 raft_consensus.cc:359] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.228668 18244 raft_consensus.cc:385] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.228919 18244 raft_consensus.cc:740] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.229708 18245 ts_tablet_manager.cc:1434] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.004s
I20260709 13:17:51.229532 18244 consensus_queue.cc:260] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.222626 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:51.233496 18090 consensus_peers.cc:597] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:51.227989 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:51.238989 18091 leader_election.cc:343] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.239660 18244 ts_tablet_manager.cc:1434] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 13:17:51.241006 18244 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:51.241609 18246 raft_consensus.cc:359] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.246110 18246 raft_consensus.cc:385] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.246672 18246 raft_consensus.cc:740] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.247385 18246 consensus_queue.cc:260] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.248895 18245 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:51.256968 18245 tablet_bootstrap.cc:654] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.256331 17874 catalog_manager.cc:5697] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.259033 18198 raft_consensus.cc:2468] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:51.266582 18247 ts_tablet_manager.cc:1434] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.080s	user 0.005s	sys 0.000s
I20260709 13:17:51.276904 18247 tablet_bootstrap.cc:492] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:51.279057 18245 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:51.268222 18246 ts_tablet_manager.cc:1434] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:17:51.279615 18245 ts_tablet_manager.cc:1403] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.031s	user 0.002s	sys 0.008s
I20260709 13:17:51.269096 18615 raft_consensus.cc:493] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.280445 18615 raft_consensus.cc:515] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.282370 18245 raft_consensus.cc:359] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.280366 18246 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
W20260709 13:17:51.269711 18090 leader_election.cc:343] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.283325 18245 raft_consensus.cc:385] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.283671 18245 raft_consensus.cc:740] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
W20260709 13:17:51.283970 18090 leader_election.cc:343] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.284441 18090 leader_election.cc:304] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:51.284341 18245 consensus_queue.cc:260] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.271147 18244 tablet_bootstrap.cc:654] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.286670 18245 ts_tablet_manager.cc:1434] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:51.287693 18245 tablet_bootstrap.cc:492] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:51.288962 18453 raft_consensus.cc:493] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.289500 18453 raft_consensus.cc:515] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.291581 18555 raft_consensus.cc:2749] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:51.293964 18245 tablet_bootstrap.cc:654] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.296211 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:51.296918 18200 raft_consensus.cc:2468] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:51.303581 18089 leader_election.cc:304] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:51.304633 18089 leader_election.cc:304] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:51.305812 18583 raft_consensus.cc:2804] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.306190 18583 raft_consensus.cc:493] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.306494 18583 raft_consensus.cc:3060] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.307762 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:51.310464 17940 leader_election.cc:343] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.309093 18246 tablet_bootstrap.cc:654] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.313630 18554 raft_consensus.cc:2804] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:51.314190 18554 raft_consensus.cc:697] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:51.299382 18247 tablet_bootstrap.cc:654] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.315016 18554 consensus_queue.cc:237] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.293860 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:51.318646 17943 leader_election.cc:343] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.319167 17943 leader_election.cc:304] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:51.320242 18489 raft_consensus.cc:2749] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:51.325119 18453 leader_election.cc:290] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:51.329699 18453 consensus_queue.cc:1048] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:51.334008 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:51.282873 18615 leader_election.cc:290] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
W20260709 13:17:51.336956 18091 leader_election.cc:343] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.349151 18453 consensus_queue.cc:1048] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:51.358937 18247 tablet_bootstrap.cc:492] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:51.359594 18247 ts_tablet_manager.cc:1403] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.000s
I20260709 13:17:51.365906 18246 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:51.366608 18246 ts_tablet_manager.cc:1403] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.000s
I20260709 13:17:51.398415 18246 raft_consensus.cc:359] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.399111 18246 raft_consensus.cc:385] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.399380 18246 raft_consensus.cc:740] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.411355 18245 tablet_bootstrap.cc:492] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:51.411983 18245 ts_tablet_manager.cc:1403] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.125s	user 0.038s	sys 0.055s
I20260709 13:17:51.414143 18246 consensus_queue.cc:260] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.417130 18247 raft_consensus.cc:359] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.417846 18247 raft_consensus.cc:385] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.418115 18247 raft_consensus.cc:740] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.419567 18247 consensus_queue.cc:260] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.420661 18245 raft_consensus.cc:359] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.421324 18245 raft_consensus.cc:385] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.421558 18245 raft_consensus.cc:740] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.422168 18245 consensus_queue.cc:260] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.424849 18247 ts_tablet_manager.cc:1434] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.001s
I20260709 13:17:51.354977 18583 raft_consensus.cc:515] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.431447 18245 ts_tablet_manager.cc:1434] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:17:51.433964 18245 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:51.435739 18246 ts_tablet_manager.cc:1434] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.004s
I20260709 13:17:51.436648 18246 tablet_bootstrap.cc:492] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:51.436811 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
W20260709 13:17:51.438270 18091 leader_election.cc:343] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.438926 18247 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:51.443535 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:51.448540 18200 raft_consensus.cc:3060] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.432986 18583 leader_election.cc:290] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:51.459322 18244 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:51.465199 18244 ts_tablet_manager.cc:1403] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.224s	user 0.012s	sys 0.005s
I20260709 13:17:51.463145 17873 catalog_manager.cc:5697] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.472926 18244 raft_consensus.cc:359] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.478929 18244 raft_consensus.cc:385] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.479243 18244 raft_consensus.cc:740] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.459941 18200 raft_consensus.cc:2468] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:51.481426 18089 leader_election.cc:304] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:51.482679 18583 raft_consensus.cc:2804] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:51.483325 18583 raft_consensus.cc:697] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:51.484162 18583 consensus_queue.cc:237] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.487656 18244 consensus_queue.cc:260] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.476928 18245 tablet_bootstrap.cc:654] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.492833 18244 ts_tablet_manager.cc:1434] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.002s
I20260709 13:17:51.493789 18244 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:51.497488 18247 tablet_bootstrap.cc:654] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.498970 18244 tablet_bootstrap.cc:654] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.468468 18246 tablet_bootstrap.cc:654] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.509197 18245 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:51.545267 18245 ts_tablet_manager.cc:1403] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.111s	user 0.010s	sys 0.001s
I20260709 13:17:51.547904 18245 raft_consensus.cc:359] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.548560 18245 raft_consensus.cc:385] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.548794 18245 raft_consensus.cc:740] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.549710 18245 consensus_queue.cc:260] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.551785 18245 ts_tablet_manager.cc:1434] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:17:51.552660 18245 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:51.555987 18244 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:51.560225 18244 ts_tablet_manager.cc:1403] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.067s	user 0.009s	sys 0.008s
I20260709 13:17:51.561662 17873 catalog_manager.cc:5697] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.562801 18244 raft_consensus.cc:359] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.563522 18244 raft_consensus.cc:385] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.569249 18245 tablet_bootstrap.cc:654] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.573138 18244 raft_consensus.cc:740] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.573997 18244 consensus_queue.cc:260] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.586328 18244 ts_tablet_manager.cc:1434] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.026s	user 0.001s	sys 0.005s
I20260709 13:17:51.594388 18244 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:51.600136 18245 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:51.600669 18245 ts_tablet_manager.cc:1403] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.000s
W20260709 13:17:51.602581 18015 consensus_peers.cc:597] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 -> Peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Couldn't send request to peer fc8a9ec31a5745d69dae4262b271e568. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:51.609328 18244 tablet_bootstrap.cc:654] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.609544 18245 raft_consensus.cc:359] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.610271 18245 raft_consensus.cc:385] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.610525 18245 raft_consensus.cc:740] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.611261 18245 consensus_queue.cc:260] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.613600 18245 ts_tablet_manager.cc:1434] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:17:51.614305 18247 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:51.614550 18245 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:51.615002 18247 ts_tablet_manager.cc:1403] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.176s	user 0.016s	sys 0.010s
I20260709 13:17:51.618044 18247 raft_consensus.cc:359] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.622373 18246 tablet_bootstrap.cc:492] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:51.622910 18246 ts_tablet_manager.cc:1403] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.032s
I20260709 13:17:51.623734 18622 raft_consensus.cc:493] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.624249 18622 raft_consensus.cc:515] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.625702 18246 raft_consensus.cc:359] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.626494 18246 raft_consensus.cc:385] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.626580 18622 leader_election.cc:290] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:51.626833 18246 raft_consensus.cc:740] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.618724 18247 raft_consensus.cc:385] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.627811 18247 raft_consensus.cc:740] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.628528 18247 consensus_queue.cc:260] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.628814 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:51.629491 18125 raft_consensus.cc:2468] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:51.630489 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:51.630791 17943 leader_election.cc:304] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:51.631733 18622 raft_consensus.cc:2804] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.632088 18622 raft_consensus.cc:493] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.632493 18622 raft_consensus.cc:3060] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.636034 18247 ts_tablet_manager.cc:1434] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 13:17:51.637101 18247 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:51.640857 18244 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:51.641367 18244 ts_tablet_manager.cc:1403] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.047s	user 0.007s	sys 0.003s
I20260709 13:17:51.643743 18246 consensus_queue.cc:260] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:51.644758 17941 leader_election.cc:343] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.645262 18244 raft_consensus.cc:359] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.646095 18244 raft_consensus.cc:385] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.646365 18244 raft_consensus.cc:740] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.648639 18622 raft_consensus.cc:515] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.650282 18557 raft_consensus.cc:493] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.650782 18557 raft_consensus.cc:515] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.652802 18557 leader_election.cc:290] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:51.653251 18247 tablet_bootstrap.cc:654] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.670543 18246 ts_tablet_manager.cc:1434] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.047s	user 0.000s	sys 0.008s
I20260709 13:17:51.671969 18244 consensus_queue.cc:260] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.673106 18246 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:51.679216 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:51.679916 18122 raft_consensus.cc:3060] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.684115 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:51.685485 18498 raft_consensus.cc:493] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.685987 18498 raft_consensus.cc:515] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.687739 18536 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.688386 18498 leader_election.cc:290] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:51.689641 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:51.690456 18244 ts_tablet_manager.cc:1434] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 13:17:51.691114 18622 leader_election.cc:290] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:51.692058 18244 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:51.694917 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:51.696640 18091 leader_election.cc:343] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.701654 18246 tablet_bootstrap.cc:654] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.702224 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:51.705649 18244 tablet_bootstrap.cc:654] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:51.708467 18089 leader_election.cc:343] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.708967 18089 leader_election.cc:304] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:51.711346 18600 raft_consensus.cc:493] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.710162 18583 raft_consensus.cc:2749] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:51.711860 18600 raft_consensus.cc:515] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.713876 18600 leader_election.cc:290] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:51.713971 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:51.719075 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "110dca2dfd484809b9e058a7ab22b49a" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:51.721350 18015 leader_election.cc:343] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.724377 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:51.728302 18246 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:51.728948 18246 ts_tablet_manager.cc:1403] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.000s
I20260709 13:17:51.733836 18246 raft_consensus.cc:359] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.734629 18246 raft_consensus.cc:385] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.734910 18246 raft_consensus.cc:740] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.736979 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "110dca2dfd484809b9e058a7ab22b49a" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:51.737723 17975 raft_consensus.cc:2468] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:51.739549 18017 leader_election.cc:304] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:51.739831 18246 consensus_queue.cc:260] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.740742 18600 raft_consensus.cc:2804] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.741178 18600 raft_consensus.cc:493] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.741577 18600 raft_consensus.cc:3060] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.741888 18615 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.742395 18615 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.745050 18615 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:51.688285 18536 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.748123 18125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:51.748935 18125 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:51.749387 18536 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:51.749913 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:51.750535 18124 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:51.751999 18166 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:51.752918 17943 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:51.759994 18615 raft_consensus.cc:2804] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.761199 18615 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.761559 18615 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.748201 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:51.764107 18200 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:51.764305 18498 consensus_queue.cc:1048] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:51.760881 18018 leader_election.cc:304] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:51.771814 18615 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.773854 18498 raft_consensus.cc:2749] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:51.774817 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:51.775473 18124 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:51.783838 17941 leader_election.cc:343] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.784296 18124 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:51.785712 17943 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:51.786597 18489 raft_consensus.cc:2804] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:51.787111 18489 raft_consensus.cc:697] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:51.788014 18489 consensus_queue.cc:237] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.797724 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:51.804337 18122 raft_consensus.cc:2468] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:51.805544 17943 leader_election.cc:304] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:51.806893 18656 raft_consensus.cc:2804] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:51.807746 18656 raft_consensus.cc:697] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:51.808730 18656 consensus_queue.cc:237] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.811153 18245 tablet_bootstrap.cc:654] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:51.819617 18166 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 67701.7us
I20260709 13:17:51.820355 18600 raft_consensus.cc:515] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.821105 18538 raft_consensus.cc:493] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.852528 18538 raft_consensus.cc:515] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.854745 18538 leader_election.cc:290] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:51.820557 18620 raft_consensus.cc:2804] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.823944 18247 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:51.855701 18247 ts_tablet_manager.cc:1403] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.219s	user 0.047s	sys 0.047s
I20260709 13:17:51.855417 18620 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.856525 18620 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.855343 18554 consensus_queue.cc:1048] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:51.839576 18244 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:51.858613 18244 ts_tablet_manager.cc:1403] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.167s	user 0.007s	sys 0.013s
I20260709 13:17:51.860733 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:51.844730 18246 ts_tablet_manager.cc:1434] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.000s
I20260709 13:17:51.848937 18583 raft_consensus.cc:493] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:51.864770 18165 leader_election.cc:343] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.866353 18246 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:51.873340 18247 raft_consensus.cc:359] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.874126 18247 raft_consensus.cc:385] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.874401 18247 raft_consensus.cc:740] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.876034 18247 consensus_queue.cc:260] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.871472 18583 raft_consensus.cc:515] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
W20260709 13:17:51.872473 18015 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 44115.1us
I20260709 13:17:51.886593 18583 leader_election.cc:290] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:51.877324 18557 raft_consensus.cc:493] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.890388 18244 raft_consensus.cc:359] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.900632 18244 raft_consensus.cc:385] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:51.900964 18244 raft_consensus.cc:740] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:51.901739 18244 consensus_queue.cc:260] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.894302 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd55fcdd2224c1db74729d68acc0ee6" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:51.902980 17975 raft_consensus.cc:2468] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:51.898247 18662 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.905030 18091 leader_election.cc:304] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:51.899227 18615 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:51.906301 18663 raft_consensus.cc:493] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.908449 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:51.912045 18247 ts_tablet_manager.cc:1434] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.008s
I20260709 13:17:51.913712 18247 tablet_bootstrap.cc:492] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:51.914595 18244 ts_tablet_manager.cc:1434] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 13:17:51.916812 18052 raft_consensus.cc:3060] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.909431 18583 raft_consensus.cc:2804] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:51.918431 18244 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:51.908869 18663 raft_consensus.cc:515] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:51.922775 18583 raft_consensus.cc:493] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.948423 18583 raft_consensus.cc:3060] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:51.910478 18620 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:51.907485 18662 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:51.907281 18557 raft_consensus.cc:515] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:51.959553 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:51.961514 18663 leader_election.cc:290] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:51.962065 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd55fcdd2224c1db74729d68acc0ee6" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:51.968299 18164 leader_election.cc:343] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.968801 18164 leader_election.cc:304] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:51.978607 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "110dca2dfd484809b9e058a7ab22b49a" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:51.984716 18246 tablet_bootstrap.cc:654] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:51.981758 18538 raft_consensus.cc:2749] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:51.996868 18245 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:51.997421 18245 ts_tablet_manager.cc:1403] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.383s	user 0.029s	sys 0.001s
I20260709 13:17:51.995420 18198 raft_consensus.cc:2393] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:51.991292 18665 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:51.994079 18015 leader_election.cc:343] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:51.996399 18600 leader_election.cc:290] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.001812 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:52.034464 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "020d6114c75749cfab1d9c0b047da996" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:52.036042 17974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "110dca2dfd484809b9e058a7ab22b49a" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:52.036854 17974 raft_consensus.cc:3060] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.039928 18662 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:52.047899 18583 raft_consensus.cc:515] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:52.049558 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:52.050926 18165 leader_election.cc:343] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.052009 17872 catalog_manager.cc:5697] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.084654 18620 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.085634 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:52.066046 18247 tablet_bootstrap.cc:654] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:52.071905 18673 log.cc:927] Time spent T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:17:52.016129 18665 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.102335 18665 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.076323 17872 catalog_manager.cc:5697] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:52.112803 18662 raft_consensus.cc:493] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.118796 18245 raft_consensus.cc:359] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.122169 18245 raft_consensus.cc:385] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.122498 18245 raft_consensus.cc:740] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.118016 18583 leader_election.cc:290] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:52.123956 18245 consensus_queue.cc:260] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.113462 18662 raft_consensus.cc:515] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
W20260709 13:17:52.138760 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 143420us
I20260709 13:17:52.147053 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:52.147760 18052 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:52.153995 18244 tablet_bootstrap.cc:654] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:52.162602 18245 ts_tablet_manager.cc:1434] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.165s	user 0.007s	sys 0.002s
I20260709 13:17:52.170379 18245 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230: Bootstrap starting.
I20260709 13:17:52.150174 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:52.176288 18017 leader_election.cc:343] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:52.197721 18091 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 203142us
I20260709 13:17:52.174137 18557 leader_election.cc:290] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:52.200683 18164 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:52.201488 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:52.204146 18200 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:52.214794 18018 leader_election.cc:304] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:52.218355 18669 raft_consensus.cc:2749] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:52.235352 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:52.235682 17972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd55fcdd2224c1db74729d68acc0ee6" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:52.243040 18245 tablet_bootstrap.cc:654] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:52.218511 18677 raft_consensus.cc:2804] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.243489 18246 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:52.245357 18246 ts_tablet_manager.cc:1403] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.379s	user 0.022s	sys 0.042s
I20260709 13:17:52.251608 17972 raft_consensus.cc:3060] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.255072 17974 raft_consensus.cc:2468] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:52.257050 18662 leader_election.cc:290] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:52.250114 18123 raft_consensus.cc:2393] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc8a9ec31a5745d69dae4262b271e568 in current term 1: Already voted for candidate 242d2387755e4a37ade4260e80f4eb6a in this term.
I20260709 13:17:52.262699 18166 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:52.246402 18677 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.264786 18677 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.266916 18244 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:52.267445 18244 ts_tablet_manager.cc:1403] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.351s	user 0.025s	sys 0.018s
W20260709 13:17:52.231545 18090 leader_election.cc:343] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.269783 18017 leader_election.cc:304] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:52.273058 18668 raft_consensus.cc:2804] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:52.273533 18668 raft_consensus.cc:697] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
W20260709 13:17:52.273738 18089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 249392us
I20260709 13:17:52.274307 18668 consensus_queue.cc:237] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.277475 18536 raft_consensus.cc:2749] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:52.278759 18247 tablet_bootstrap.cc:492] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:52.279289 18247 ts_tablet_manager.cc:1403] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.366s	user 0.013s	sys 0.016s
I20260709 13:17:52.279738 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb77b44e31244ec29dad965bc258249a" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:52.280411 18123 raft_consensus.cc:2468] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 0.
I20260709 13:17:52.282053 18247 raft_consensus.cc:359] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:52.282795 18247 raft_consensus.cc:385] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.283037 18247 raft_consensus.cc:740] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.280821 18246 raft_consensus.cc:359] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.296169 18246 raft_consensus.cc:385] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.296543 18246 raft_consensus.cc:740] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.297327 18246 consensus_queue.cc:260] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.297508 18247 consensus_queue.cc:260] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:52.300122 18247 ts_tablet_manager.cc:1434] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:17:52.303519 18247 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:52.303316 18244 raft_consensus.cc:359] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.303778 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:52.304136 18244 raft_consensus.cc:385] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.304471 18244 raft_consensus.cc:740] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.305444 18246 ts_tablet_manager.cc:1434] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 13:17:52.306423 18246 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:52.304430 18200 raft_consensus.cc:2468] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
W20260709 13:17:52.312420 18166 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 49842.3us
I20260709 13:17:52.322144 18246 tablet_bootstrap.cc:654] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:52.323562 18247 tablet_bootstrap.cc:654] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:52.329864 18245 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230: No bootstrap required, opened a new log
I20260709 13:17:52.330440 18245 ts_tablet_manager.cc:1403] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.007s
W20260709 13:17:52.332526 17943 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 50282.1us
I20260709 13:17:52.335995 18655 raft_consensus.cc:493] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.336514 18655 raft_consensus.cc:515] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.338474 18655 leader_election.cc:290] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
W20260709 13:17:52.343582 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 142974us
W20260709 13:17:52.348433 18090 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 120404us
I20260709 13:17:52.332008 18677 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.319199 17972 raft_consensus.cc:2468] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
W20260709 13:17:52.385568 18018 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 45526.8us
I20260709 13:17:52.313278 18244 consensus_queue.cc:260] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.381753 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:52.385241 18090 consensus_peers.cc:597] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:52.386854 18200 raft_consensus.cc:2468] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:52.394423 18166 leader_election.cc:304] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:52.395098 18015 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:52.396749 18244 ts_tablet_manager.cc:1434] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.003s
I20260709 13:17:52.400668 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb77b44e31244ec29dad965bc258249a" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
W20260709 13:17:52.411562 18164 leader_election.cc:343] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.412568 18244 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:52.411070 18091 leader_election.cc:304] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:52.415479 18245 raft_consensus.cc:359] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:52.416229 18245 raft_consensus.cc:385] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.416483 18245 raft_consensus.cc:740] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.417199 18245 consensus_queue.cc:260] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:52.429785 18680 raft_consensus.cc:2804] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:52.430469 18672 raft_consensus.cc:2804] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.430887 18672 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.431205 18672 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.418279 18538 raft_consensus.cc:2804] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.435338 18245 ts_tablet_manager.cc:1434] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.000s
I20260709 13:17:52.447247 18089 leader_election.cc:304] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:52.448372 18557 raft_consensus.cc:2804] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.448773 18557 raft_consensus.cc:493] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.449102 18557 raft_consensus.cc:3060] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.439878 18615 consensus_queue.cc:1048] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:17:52.440251 18538 raft_consensus.cc:493] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.456815 18538 raft_consensus.cc:3060] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.457360 18557 raft_consensus.cc:515] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:52.459524 18557 leader_election.cc:290] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:52.463213 18052 raft_consensus.cc:3060] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.446162 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:52.467276 18126 raft_consensus.cc:2468] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:52.435303 18680 raft_consensus.cc:697] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:52.432843 18453 consensus_queue.cc:1048] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 13:17:52.484124 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:52.485391 18049 raft_consensus.cc:2468] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:52.484519 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd55fcdd2224c1db74729d68acc0ee6" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:52.489025 18680 consensus_queue.cc:237] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:52.476357 18622 consensus_queue.cc:1048] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
W20260709 13:17:52.510749 18091 consensus_peers.cc:597] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:52.515623 18051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:52.516263 18051 raft_consensus.cc:3060] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.520645 17974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:52.522852 18051 raft_consensus.cc:2468] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:52.523183 18677 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:52.532213 17941 consensus_peers.cc:597] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:52.533027 18165 leader_election.cc:343] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.533793 18672 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.535107 18246 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:52.535748 18246 ts_tablet_manager.cc:1403] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.230s	user 0.022s	sys 0.000s
I20260709 13:17:52.538763 18246 raft_consensus.cc:359] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.539597 18246 raft_consensus.cc:385] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.539934 18246 raft_consensus.cc:740] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.540637 18246 consensus_queue.cc:260] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.541990 18247 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:52.542512 18247 ts_tablet_manager.cc:1403] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.239s	user 0.017s	sys 0.003s
I20260709 13:17:52.542687 18538 raft_consensus.cc:515] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:52.512729 18693 raft_consensus.cc:493] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.545814 18693 raft_consensus.cc:515] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:52.547988 18693 leader_election.cc:290] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:52.551131 18089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 58659.7us
I20260709 13:17:52.557358 18246 ts_tablet_manager.cc:1434] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:17:52.558255 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:52.558801 18246 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
W20260709 13:17:52.503861 17940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 59337.3us
I20260709 13:17:52.559068 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:52.561132 17975 raft_consensus.cc:2468] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:52.558889 18197 raft_consensus.cc:3060] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.567700 17940 leader_election.cc:304] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
W20260709 13:17:52.557300 18166 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 44251.8us
I20260709 13:17:52.568477 18244 tablet_bootstrap.cc:654] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:52.568519 18672 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.569948 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:52.584710 18195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:52.585621 18195 raft_consensus.cc:2393] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
I20260709 13:17:52.586372 18689 raft_consensus.cc:2804] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.586804 18689 raft_consensus.cc:493] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.587122 18689 raft_consensus.cc:3060] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.591269 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:52.592221 18049 raft_consensus.cc:2393] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc8a9ec31a5745d69dae4262b271e568 in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
W20260709 13:17:52.572045 18090 leader_election.cc:343] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.574913 18677 raft_consensus.cc:493] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.596503 18677 raft_consensus.cc:515] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.597519 18164 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:52.598651 18677 leader_election.cc:290] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:52.599182 18620 raft_consensus.cc:2749] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:52.599625 17975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:52.607455 18091 leader_election.cc:304] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:52.612449 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:52.613144 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f2c37906b94d31b85748436677deef" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
W20260709 13:17:52.616508 18017 leader_election.cc:343] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.618481 18017 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:52.619549 18672 raft_consensus.cc:2749] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:52.651932 18247 raft_consensus.cc:359] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.652845 18247 raft_consensus.cc:385] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.653122 18247 raft_consensus.cc:740] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
W20260709 13:17:52.656872 18091 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 49527.9us
I20260709 13:17:52.622884 18689 raft_consensus.cc:515] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.659300 18453 raft_consensus.cc:2804] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.664873 18247 consensus_queue.cc:260] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.674227 18164 leader_election.cc:304] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
W20260709 13:17:52.683614 18090 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 73459.8us
I20260709 13:17:52.684370 18090 leader_election.cc:304] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
W20260709 13:17:52.686184 18090 leader_election.cc:343] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.688887 18583 raft_consensus.cc:2804] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:52.689345 18583 raft_consensus.cc:697] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:52.690130 18583 consensus_queue.cc:237] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:52.695003 18246 tablet_bootstrap.cc:654] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:52.695876 18662 raft_consensus.cc:2749] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:52.685066 18247 ts_tablet_manager.cc:1434] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.142s	user 0.007s	sys 0.001s
I20260709 13:17:52.706171 18689 leader_election.cc:290] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:52.707648 18538 leader_election.cc:290] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:52.708477 18555 raft_consensus.cc:493] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.708966 18555 raft_consensus.cc:515] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.671147 18197 raft_consensus.cc:2468] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:52.671492 18453 raft_consensus.cc:493] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.711643 18453 raft_consensus.cc:3060] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.712806 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb77b44e31244ec29dad965bc258249a" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:52.713845 18195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:52.714818 18164 leader_election.cc:343] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:52.715531 18089 leader_election.cc:343] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.727985 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:52.710951 18555 leader_election.cc:290] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.728710 17976 raft_consensus.cc:2468] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 0.
I20260709 13:17:52.729820 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb77b44e31244ec29dad965bc258249a" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:52.730417 18123 raft_consensus.cc:3060] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.732321 18247 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:52.722687 18697 raft_consensus.cc:493] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.746683 18091 leader_election.cc:304] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:52.751113 18244 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:52.751715 18244 ts_tablet_manager.cc:1403] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.341s	user 0.039s	sys 0.016s
I20260709 13:17:52.743847 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:52.764530 18124 raft_consensus.cc:3060] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.766876 18247 tablet_bootstrap.cc:654] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:52.769706 18246 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:52.770220 18246 ts_tablet_manager.cc:1403] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.212s	user 0.016s	sys 0.004s
I20260709 13:17:52.755914 18656 consensus_queue.cc:1048] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:52.777910 18672 raft_consensus.cc:493] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.778422 18672 raft_consensus.cc:515] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.753067 18697 raft_consensus.cc:515] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.755307 18583 raft_consensus.cc:2804] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:52.772698 18015 consensus_peers.cc:597] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 -> Peer fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171): Couldn't send request to peer fc8a9ec31a5745d69dae4262b271e568. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:52.782663 18123 raft_consensus.cc:2468] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:52.784168 18124 raft_consensus.cc:2468] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:52.790347 18244 raft_consensus.cc:359] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.782395 18583 raft_consensus.cc:493] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.791791 18244 raft_consensus.cc:385] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.793082 18244 raft_consensus.cc:740] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.794674 18244 consensus_queue.cc:260] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.801961 18195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:52.802678 18195 raft_consensus.cc:2468] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:52.803622 18166 leader_election.cc:304] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:52.804898 18685 raft_consensus.cc:2804] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:52.805378 18685 raft_consensus.cc:697] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
W20260709 13:17:52.808650 18705 log.cc:927] Time spent T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:17:52.808640 18453 raft_consensus.cc:515] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:52.794201 18583 raft_consensus.cc:3060] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.814992 18244 ts_tablet_manager.cc:1434] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.002s
I20260709 13:17:52.816005 18244 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:17:52.806221 18685 consensus_queue.cc:237] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:52.816869 18015 leader_election.cc:304] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:52.818758 18666 raft_consensus.cc:2804] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.819123 18666 raft_consensus.cc:493] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.819242 18200 raft_consensus.cc:3060] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.819501 18666 raft_consensus.cc:3060] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.825168 18453 leader_election.cc:290] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:52.797842 18672 leader_election.cc:290] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.835137 18697 leader_election.cc:290] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:52.845655 18666 raft_consensus.cc:515] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:52.846715 18583 raft_consensus.cc:515] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.850541 18583 leader_election.cc:290] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.852581 18246 raft_consensus.cc:359] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.853428 18246 raft_consensus.cc:385] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:52.853699 18246 raft_consensus.cc:740] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:52.854420 18246 consensus_queue.cc:260] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.856447 18666 leader_election.cc:290] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:52.902529 17943 leader_election.cc:304] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:17:52.917766 18195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:52.919759 18246 ts_tablet_manager.cc:1434] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.149s	user 0.008s	sys 0.000s
I20260709 13:17:52.922863 18246 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568: Bootstrap starting.
I20260709 13:17:52.922576 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:52.925956 18194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:52.917464 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:52.928354 18050 raft_consensus.cc:2393] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:52.919212 18195 raft_consensus.cc:3060] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.930287 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:52.931360 17973 raft_consensus.cc:2468] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 0.
I20260709 13:17:52.926397 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:52.935963 18196 raft_consensus.cc:3060] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.940474 18659 raft_consensus.cc:2804] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:52.926959 18194 raft_consensus.cc:2468] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 0.
I20260709 13:17:52.961005 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:52.962149 18198 raft_consensus.cc:3060] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.941386 18659 raft_consensus.cc:697] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:17:52.967357 17977 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:52.970278 17977 raft_consensus.cc:3060] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:52.976720 17940 leader_election.cc:304] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:52.978256 18656 raft_consensus.cc:2804] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:52.978665 18656 raft_consensus.cc:493] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.979027 18656 raft_consensus.cc:3060] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:52.981565 18090 leader_election.cc:343] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:52.985070 18247 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:52.986196 18247 ts_tablet_manager.cc:1403] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.254s	user 0.013s	sys 0.006s
I20260709 13:17:52.992365 18672 consensus_queue.cc:1048] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:53.010558 17972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:53.011001 17976 raft_consensus.cc:3060] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:53.013562 17972 raft_consensus.cc:3060] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:53.015125 18659 consensus_queue.cc:237] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:52.990839 18247 raft_consensus.cc:359] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:53.021245 18246 tablet_bootstrap.cc:654] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:53.021786 18247 raft_consensus.cc:385] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:53.024510 18247 raft_consensus.cc:740] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:53.025203 17973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:53.037348 18696 consensus_queue.cc:1048] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:17:53.044018 18715 log.cc:927] Time spent T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:17:53.046697 17972 raft_consensus.cc:2468] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:53.052012 18244 tablet_bootstrap.cc:654] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:53.057123 18196 raft_consensus.cc:2468] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:53.028954 18247 consensus_queue.cc:260] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:53.061547 18089 leader_election.cc:304] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:53.062680 18247 ts_tablet_manager.cc:1434] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.076s	user 0.000s	sys 0.007s
I20260709 13:17:53.068411 18696 raft_consensus.cc:2804] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:53.069057 18696 raft_consensus.cc:697] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:53.070236 18696 consensus_queue.cc:237] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:53.076494 18049 raft_consensus.cc:3060] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:53.085289 18090 consensus_peers.cc:597] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:53.139310 17977 raft_consensus.cc:2468] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:53.122892 17875 catalog_manager.cc:5697] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.161602 18244 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: No bootstrap required, opened a new log
I20260709 13:17:53.162545 18244 ts_tablet_manager.cc:1403] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.347s	user 0.009s	sys 0.007s
I20260709 13:17:53.164815 18656 raft_consensus.cc:515] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:53.174217 18247 tablet_bootstrap.cc:492] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:53.175601 17973 raft_consensus.cc:2393] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 1: Already voted for candidate 242d2387755e4a37ade4260e80f4eb6a in this term.
I20260709 13:17:53.183809 18091 leader_election.cc:304] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272
I20260709 13:17:53.190608 18197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:53.198026 18453 raft_consensus.cc:2804] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:53.198747 18453 raft_consensus.cc:697] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:53.199741 18453 consensus_queue.cc:237] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:53.210800 18195 raft_consensus.cc:2468] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 1.
I20260709 13:17:53.210983 18555 consensus_queue.cc:1048] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:17:53.233525 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:53.214308 18244 raft_consensus.cc:359] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:53.263626 18244 raft_consensus.cc:385] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:53.264061 18244 raft_consensus.cc:740] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:17:53.247920 18198 raft_consensus.cc:2468] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:53.286947 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bd29ab75004ddf9b1ce255a071ed24" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:53.291391 18244 consensus_queue.cc:260] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:53.273546 18656 leader_election.cc:290] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:53.314222 18622 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.314930 18622 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:53.317448 18622 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:53.321841 18015 leader_election.cc:304] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:53.312237 18047 raft_consensus.cc:2393] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:53.330013 18244 ts_tablet_manager.cc:1434] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.167s	user 0.011s	sys 0.000s
I20260709 13:17:53.332370 18665 raft_consensus.cc:2804] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:53.333127 18665 raft_consensus.cc:697] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:53.334147 18665 consensus_queue.cc:237] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
W20260709 13:17:53.336305 17940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 45231.5us
I20260709 13:17:53.340294 17874 catalog_manager.cc:5697] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 1 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.341789 18195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:53.343070 18195 raft_consensus.cc:2393] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fc8a9ec31a5745d69dae4262b271e568 in this term.
W20260709 13:17:53.362643 18017 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 66582.8us
W20260709 13:17:53.363706 18164 consensus_peers.cc:597] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:53.364341 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 64249.3us
I20260709 13:17:53.365296 18660 consensus_queue.cc:1048] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:53.367178 17941 leader_election.cc:304] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:53.370278 18246 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568: No bootstrap required, opened a new log
I20260709 13:17:53.370959 18246 ts_tablet_manager.cc:1403] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568: Time spent bootstrapping tablet: real 0.449s	user 0.021s	sys 0.033s
I20260709 13:17:53.374652 18615 raft_consensus.cc:2749] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:53.377045 18247 tablet_bootstrap.cc:654] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:53.388303 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:53.389338 18047 raft_consensus.cc:2393] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 242d2387755e4a37ade4260e80f4eb6a in current term 1: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:53.393741 18683 consensus_queue.cc:1048] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260709 13:17:53.452657 17941 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a; no voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:53.451337 18246 raft_consensus.cc:359] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:53.455080 18246 raft_consensus.cc:385] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:53.455478 18246 raft_consensus.cc:740] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Initialized, Role: FOLLOWER
I20260709 13:17:53.456085 18246 consensus_queue.cc:260] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:53.476136 18718 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:53.499015 18718 raft_consensus.cc:2749] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:53.509479 17875 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47122 (request call id 86) took 1089 ms. Trace:
I20260709 13:17:53.505625 18697 consensus_queue.cc:1048] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:53.511302 18246 ts_tablet_manager.cc:1434] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568: Time spent starting tablet: real 0.140s	user 0.012s	sys 0.000s
I20260709 13:17:53.509935 17875 rpcz_store.cc:276] 0709 13:17:52.432018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.434016 (+  1998us) service_pool.cc:224] Handling call
0709 13:17:53.509391 (+1075375us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:53.279798 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:53.280439 (+   641us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:53.280484 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:17:53.280516 (+    32us) tablet.cc:662] Decoding operations
0709 13:17:53.280927 (+   411us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:53.281012 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:53.281041 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:53.281329 (+   288us) tablet.cc:701] Row locks acquired
0709 13:17:53.281361 (+    32us) write_op.cc:260] PREPARE: finished
0709 13:17:53.281515 (+   154us) write_op.cc:270] Start()
0709 13:17:53.281681 (+   166us) write_op.cc:276] Timestamp: P: 1783603073281451 usec, L: 0
0709 13:17:53.281714 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:17:53.282473 (+   759us) log.cc:844] Serialized 534 byte log entry
0709 13:17:53.311757 (+ 29284us) op_driver.cc:464] REPLICATION: finished
0709 13:17:53.499821 (+188064us) write_op.cc:301] APPLY: starting
0709 13:17:53.499887 (+    66us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:53.499970 (+    83us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:53.500074 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:53.500366 (+   292us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:53.500562 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:17:53.500603 (+    41us) write_op.cc:312] APPLY: finished
0709 13:17:53.500813 (+   210us) log.cc:844] Serialized 22 byte log entry
0709 13:17:53.500977 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:53.501048 (+    71us) write_op.cc:454] Released schema lock
0709 13:17:53.501104 (+    56us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":187799,"mutex_wait_us":32539,"num_ops":1,"prepare.queue_time_us":118128,"prepare.run_cpu_time_us":3099,"prepare.run_wall_time_us":14510,"replication_time_us":29882,"thread_start_us":232423,"threads_started":2}]]}
I20260709 13:17:53.543536 18659 consensus_queue.cc:1048] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:53.566857 18554 consensus_queue.cc:1048] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:53.598140 17872 catalog_manager.cc:5697] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.600178 17872 catalog_manager.cc:5697] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:53.602587 17872 catalog_manager.cc:5697] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.604880 17872 catalog_manager.cc:5697] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:53.639904 18247 tablet_bootstrap.cc:492] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:53.640479 18247 ts_tablet_manager.cc:1403] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.467s	user 0.016s	sys 0.015s
I20260709 13:17:53.641896 18453 consensus_queue.cc:1048] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:53.643441 18247 raft_consensus.cc:359] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:53.644248 18247 raft_consensus.cc:385] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:53.644503 18247 raft_consensus.cc:740] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:53.645282 18247 consensus_queue.cc:260] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:53.648698 18247 ts_tablet_manager.cc:1434] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:53.653643 18247 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:53.680331 18247 tablet_bootstrap.cc:654] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:53.707334 18700 consensus_queue.cc:1048] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:53.725672 18196 raft_consensus.cc:3060] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:53.739434 17874 catalog_manager.cc:5697] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.758963 17873 catalog_manager.cc:5697] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 1 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.768580 18247 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:53.769117 18247 ts_tablet_manager.cc:1403] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.116s	user 0.009s	sys 0.007s
W20260709 13:17:53.779995 18090 consensus_peers.cc:597] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:53.813297 18247 raft_consensus.cc:359] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:53.814114 18247 raft_consensus.cc:385] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:53.814402 18247 raft_consensus.cc:740] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:53.825721 18247 consensus_queue.cc:260] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:53.828069 18247 ts_tablet_manager.cc:1434] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
I20260709 13:17:53.832265 18247 tablet_bootstrap.cc:492] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:53.845469 18047 raft_consensus.cc:3060] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:53.895941 18247 tablet_bootstrap.cc:654] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:53.915546 18247 tablet_bootstrap.cc:492] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:53.916013 18247 ts_tablet_manager.cc:1403] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.084s	user 0.002s	sys 0.014s
I20260709 13:17:53.918887 18247 raft_consensus.cc:359] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:53.919554 18247 raft_consensus.cc:385] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:53.919803 18247 raft_consensus.cc:740] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:53.920557 18247 consensus_queue.cc:260] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:53.926237 18247 ts_tablet_manager.cc:1434] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:17:53.929549 18666 consensus_queue.cc:1048] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:53.933486 18247 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272: Bootstrap starting.
I20260709 13:17:53.942131 18247 tablet_bootstrap.cc:654] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:53.969991 18666 consensus_queue.cc:1048] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:53.973670 18666 raft_consensus.cc:493] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.974107 18666 raft_consensus.cc:515] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:53.977438 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691cfc20fdd444b3b53db6fdcdedcda8" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:53.977927 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "691cfc20fdd444b3b53db6fdcdedcda8" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:53.980582 18018 leader_election.cc:304] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:53.981750 18665 raft_consensus.cc:2749] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:53.983306 18666 leader_election.cc:290] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:54.016700 18047 raft_consensus.cc:3060] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:54.053326 18247 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272: No bootstrap required, opened a new log
I20260709 13:17:54.053911 18247 ts_tablet_manager.cc:1403] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272: Time spent bootstrapping tablet: real 0.121s	user 0.022s	sys 0.040s
I20260709 13:17:54.058919 18247 raft_consensus.cc:359] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:54.059625 18247 raft_consensus.cc:385] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:54.060148 18247 raft_consensus.cc:740] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Initialized, Role: FOLLOWER
I20260709 13:17:54.066392 18247 consensus_queue.cc:260] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:54.068583 18247 ts_tablet_manager.cc:1434] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:17:54.148542 18666 raft_consensus.cc:493] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.149053 18666 raft_consensus.cc:515] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:54.151058 18666 leader_election.cc:290] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:54.152504 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:54.154626 17976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:54.156276 18017 leader_election.cc:304] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:54.157020 18666 raft_consensus.cc:2749] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:54.233771 18047 raft_consensus.cc:3060] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:54.338536 17923 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:54.360121 18047 raft_consensus.cc:3060] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:54.548730 18660 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.549364 18660 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:54.554368 17970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a" is_pre_election: true
I20260709 13:17:54.556763 17970 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:54.556933 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:54.552085 18660 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:54.558214 18050 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:54.559545 18165 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:54.560346 18660 raft_consensus.cc:2804] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:54.560750 18660 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.561082 18660 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:54.574640 18660 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:54.577042 18660 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:54.578118 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:54.579196 18050 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:54.579716 17970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:17:54.580333 17970 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:54.588382 17970 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:54.589807 18165 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:54.590761 18660 raft_consensus.cc:2804] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:54.591272 18660 raft_consensus.cc:697] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:54.592142 18660 consensus_queue.cc:237] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:54.618515 18050 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:54.638999 17871 catalog_manager.cc:5697] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 0 to 2, leader changed from <none> to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:55.152601 18047 raft_consensus.cc:1275] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:55.154214 18660 consensus_queue.cc:1048] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:55.201519 17976 raft_consensus.cc:1275] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:55.205770 18660 consensus_queue.cc:1048] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:55.734167 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
W20260709 13:17:55.770277 18017 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20260709 13:17:55.785965 18165 consensus_peers.cc:597] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.792225 18091 consensus_peers.cc:597] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.801213 18091 consensus_peers.cc:597] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.803382 18165 consensus_peers.cc:597] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.820428 18017 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 50180.2us
W20260709 13:17:55.823004 18091 consensus_peers.cc:597] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.823693 18017 consensus_peers.cc:597] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.857268 18091 consensus_peers.cc:597] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.860575 18017 consensus_peers.cc:597] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.861364 18017 consensus_peers.cc:597] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.883252 18091 consensus_peers.cc:597] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.883581 18017 consensus_peers.cc:597] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.884910 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:55.885855 10257 tablet_replica.cc:333] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.886726 10257 raft_consensus.cc:2243] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.887979 10257 raft_consensus.cc:2272] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.896852 10257 tablet_replica.cc:333] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.897900 10257 raft_consensus.cc:2243] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.898607 10257 raft_consensus.cc:2272] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.906697 10257 tablet_replica.cc:333] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.908833 10257 raft_consensus.cc:2243] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.909837 10257 raft_consensus.cc:2272] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.912900 10257 tablet_replica.cc:333] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.913753 10257 raft_consensus.cc:2243] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.914263 10257 raft_consensus.cc:2272] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.916286 10257 tablet_replica.cc:333] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.916982 10257 raft_consensus.cc:2243] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.917389 10257 raft_consensus.cc:2272] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.919737 10257 tablet_replica.cc:333] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.920315 10257 raft_consensus.cc:2243] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.920851 10257 raft_consensus.cc:2272] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.922894 10257 tablet_replica.cc:333] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.923542 10257 raft_consensus.cc:2243] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.923902 10257 raft_consensus.cc:2272] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.926134 10257 tablet_replica.cc:333] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.926887 10257 raft_consensus.cc:2243] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.927378 10257 raft_consensus.cc:2272] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.930178 10257 tablet_replica.cc:333] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.930751 10257 raft_consensus.cc:2243] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.931986 10257 raft_consensus.cc:2272] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.934772 10257 tablet_replica.cc:333] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.935611 10257 raft_consensus.cc:2243] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:55.935851 18091 consensus_peers.cc:597] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.936101 10257 raft_consensus.cc:2272] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.938944 10257 tablet_replica.cc:333] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.940049 10257 raft_consensus.cc:2243] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.940600 10257 raft_consensus.cc:2272] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.943820 10257 tablet_replica.cc:333] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.944655 10257 raft_consensus.cc:2243] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.945173 10257 raft_consensus.cc:2272] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.948086 10257 tablet_replica.cc:333] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.948954 10257 raft_consensus.cc:2243] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.950045 10257 raft_consensus.cc:2272] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.952879 10257 tablet_replica.cc:333] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.953719 10257 raft_consensus.cc:2243] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.954264 10257 raft_consensus.cc:2272] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.956936 10257 tablet_replica.cc:333] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.957768 10257 raft_consensus.cc:2243] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.958305 10257 raft_consensus.cc:2272] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.961110 10257 tablet_replica.cc:333] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
W20260709 13:17:55.961679 18091 consensus_peers.cc:597] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.961879 10257 raft_consensus.cc:2243] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.965397 10257 raft_consensus.cc:2272] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.968365 10257 tablet_replica.cc:333] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.969133 10257 raft_consensus.cc:2243] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.970191 10257 raft_consensus.cc:2272] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.973081 10257 tablet_replica.cc:333] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.973840 10257 raft_consensus.cc:2243] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.974889 10257 raft_consensus.cc:2272] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.977524 10257 tablet_replica.cc:333] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.978253 10257 raft_consensus.cc:2243] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.979255 10257 raft_consensus.cc:2272] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.982460 10257 tablet_replica.cc:333] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.983197 10257 raft_consensus.cc:2243] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.983713 10257 raft_consensus.cc:2272] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:55.987352 18017 consensus_peers.cc:597] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:55.990628 18165 consensus_peers.cc:597] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.991183 10257 tablet_replica.cc:333] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:55.992041 10257 raft_consensus.cc:2243] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:55.993099 10257 raft_consensus.cc:2272] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:55.995852 10257 tablet_replica.cc:333] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
W20260709 13:17:55.995549 18165 consensus_peers.cc:597] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.996773 10257 raft_consensus.cc:2243] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.997344 10257 raft_consensus.cc:2272] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:55.997730 18165 consensus_peers.cc:597] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.999581 10257 tablet_replica.cc:333] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.000151 10257 raft_consensus.cc:2243] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.000569 10257 raft_consensus.cc:2272] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.003373 10257 tablet_replica.cc:333] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.004125 10257 raft_consensus.cc:2243] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.004822 10257 raft_consensus.cc:2272] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.007685 10257 tablet_replica.cc:333] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.008422 10257 raft_consensus.cc:2243] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.008921 10257 raft_consensus.cc:2272] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.011296 10257 tablet_replica.cc:333] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.011988 10257 raft_consensus.cc:2243] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.012497 10257 raft_consensus.cc:2272] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.030170 10257 tablet_replica.cc:333] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.031046 10257 raft_consensus.cc:2243] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.035606 10257 raft_consensus.cc:2272] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.038707 10257 tablet_replica.cc:333] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.039520 10257 raft_consensus.cc:2243] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.040025 10257 raft_consensus.cc:2272] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:56.041147 18165 consensus_peers.cc:597] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:56.042680 10257 tablet_replica.cc:333] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.044005 10257 raft_consensus.cc:2243] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.044687 10257 raft_consensus.cc:2272] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.047485 10257 tablet_replica.cc:333] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.048224 10257 raft_consensus.cc:2243] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.048704 10257 raft_consensus.cc:2272] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.051551 10257 tablet_replica.cc:333] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.052294 10257 raft_consensus.cc:2243] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.052776 10257 raft_consensus.cc:2272] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.055851 10257 tablet_replica.cc:333] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.056850 10257 raft_consensus.cc:2243] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.057554 10257 raft_consensus.cc:2272] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:56.059263 18091 consensus_peers.cc:597] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:56.060348 10257 tablet_replica.cc:333] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.061172 10257 raft_consensus.cc:2243] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:56.062330 10257 raft_consensus.cc:2272] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.065114 10257 tablet_replica.cc:333] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.065801 10257 raft_consensus.cc:2243] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:56.067042 10257 raft_consensus.cc:2272] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.069907 10257 tablet_replica.cc:333] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.070679 10257 raft_consensus.cc:2243] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:56.071698 10257 raft_consensus.cc:2272] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.074353 10257 tablet_replica.cc:333] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.075076 10257 raft_consensus.cc:2243] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.075659 10257 raft_consensus.cc:2272] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.078524 10257 tablet_replica.cc:333] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.079265 10257 raft_consensus.cc:2243] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:56.080252 10257 raft_consensus.cc:2272] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.083097 10257 tablet_replica.cc:333] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.083829 10257 raft_consensus.cc:2243] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.084390 10257 raft_consensus.cc:2272] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.087304 10257 tablet_replica.cc:333] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.088001 10257 raft_consensus.cc:2243] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.088459 10257 raft_consensus.cc:2272] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:56.090363 18091 consensus_peers.cc:597] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:56.093874 10257 tablet_replica.cc:333] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.094666 10257 raft_consensus.cc:2243] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.095224 10257 raft_consensus.cc:2272] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.098156 10257 tablet_replica.cc:333] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.098872 10257 raft_consensus.cc:2243] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.099439 10257 raft_consensus.cc:2272] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.102555 10257 tablet_replica.cc:333] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.103245 10257 raft_consensus.cc:2243] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:56.104269 10257 raft_consensus.cc:2272] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.107043 10257 tablet_replica.cc:333] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.107718 10257 raft_consensus.cc:2243] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.108240 10257 raft_consensus.cc:2272] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.110801 10257 tablet_replica.cc:333] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.111747 10257 raft_consensus.cc:2243] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:56.112505 10257 raft_consensus.cc:2272] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.115075 10257 tablet_replica.cc:333] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: stopping tablet replica
I20260709 13:17:56.116781 10257 raft_consensus.cc:2243] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.117306 10257 raft_consensus.cc:2272] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:56.119184 18091 consensus_peers.cc:597] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.120537 18091 consensus_peers.cc:597] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.123844 18091 consensus_peers.cc:597] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.132141 18165 consensus_peers.cc:597] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.152860 18091 consensus_peers.cc:597] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.155503 18091 consensus_peers.cc:597] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.157567 18165 consensus_peers.cc:597] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.197458 18165 consensus_peers.cc:597] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.204241 18091 consensus_peers.cc:597] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.281629 18017 consensus_peers.cc:597] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.289968 18017 consensus_peers.cc:597] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:56.427685 18090 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 90185.6us
I20260709 13:17:56.908082 18779 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:56.908633 18779 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:56.910691 18779 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:56.912673 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:56.916621 18091 leader_election.cc:336] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:56.917030 18091 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:56.917716 18779 raft_consensus.cc:2749] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:56.923738 18685 raft_consensus.cc:493] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:56.924377 18685 raft_consensus.cc:515] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:56.926367 18685 leader_election.cc:290] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:56.932371 18165 leader_election.cc:336] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:56.934783 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:56.943531 18164 leader_election.cc:304] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:56.944909 18536 raft_consensus.cc:2749] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:56.958622 18536 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:56.959095 18536 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:56.961798 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:56.962433 18123 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:56.963850 18166 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:56.964727 18776 raft_consensus.cc:2804] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:56.965082 18776 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:56.965425 18776 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:56.970744 18536 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:56.975129 18165 leader_election.cc:336] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:56.984607 18776 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:56.987042 18776 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:56.993407 18165 leader_election.cc:336] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.010489 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:57.011132 18123 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.019500 18123 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:57.022212 18784 raft_consensus.cc:493] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.021397 18166 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.022822 18784 raft_consensus.cc:515] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.023370 18536 raft_consensus.cc:2804] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.023931 18536 raft_consensus.cc:697] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:57.024668 18536 consensus_queue.cc:237] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.052027 18785 raft_consensus.cc:493] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.052551 18785 raft_consensus.cc:515] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.054857 18785 leader_election.cc:290] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.058748 18780 raft_consensus.cc:493] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.062394 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7187dfc4efc34078a81a1f59637521c8" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
W20260709 13:17:57.067314 18165 leader_election.cc:336] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.059330 18780 raft_consensus.cc:515] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.073278 17871 catalog_manager.cc:5697] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.078132 18780 leader_election.cc:290] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:57.079741 18165 leader_election.cc:336] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.083634 18165 leader_election.cc:304] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.084609 18785 raft_consensus.cc:2749] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.088398 18194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.089279 18784 leader_election.cc:290] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.090858 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:57.094077 18164 leader_election.cc:304] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:57.095032 18785 raft_consensus.cc:2749] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:57.095920 18091 leader_election.cc:336] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.097946 18091 leader_election.cc:304] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:57.101090 18780 raft_consensus.cc:2749] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.116499 18780 raft_consensus.cc:493] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.117038 18780 raft_consensus.cc:515] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.120700 18665 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.121346 18665 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.119239 18780 leader_election.cc:290] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.124990 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.126365 18781 raft_consensus.cc:493] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.126888 18781 raft_consensus.cc:515] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.130852 18780 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.131227 18781 leader_election.cc:290] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:57.131937 18017 leader_election.cc:336] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.132470 18091 leader_election.cc:336] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.131422 18780 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.132750 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.134336 18665 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:57.136364 18091 leader_election.cc:304] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:57.137179 18779 raft_consensus.cc:2749] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.138259 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:57.139063 18049 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:57.142763 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:57.143448 18123 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
W20260709 13:17:57.145582 18091 leader_election.cc:336] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.146992 18090 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.148883 18018 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.149725 18781 raft_consensus.cc:2804] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.150126 18781 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.150275 18782 raft_consensus.cc:2804] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.150527 18781 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.153175 18782 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.153582 18782 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.160841 18781 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.163105 18781 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:57.164005 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:57.164919 18780 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.167600 18665 raft_consensus.cc:493] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.168238 18665 raft_consensus.cc:515] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.170116 18665 leader_election.cc:290] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:57.167670 18017 leader_election.cc:336] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.171746 18017 leader_election.cc:304] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:57.172068 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.172389 18781 raft_consensus.cc:2749] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.172757 18196 raft_consensus.cc:2468] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:57.176589 18015 leader_election.cc:304] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.177758 18781 raft_consensus.cc:2804] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.178112 18781 raft_consensus.cc:493] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.178447 18781 raft_consensus.cc:3060] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.190955 18785 raft_consensus.cc:493] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.191591 18785 raft_consensus.cc:515] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.193554 18785 leader_election.cc:290] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.195797 18781 raft_consensus.cc:515] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
W20260709 13:17:57.197098 10257 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 17998)
W20260709 13:17:57.200095 18165 leader_election.cc:336] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.199944 18194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211547823b99438c9cc10432cc919a5f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:57.201014 18194 raft_consensus.cc:3060] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.203521 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:57.204182 18050 raft_consensus.cc:2468] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:57.206961 18164 leader_election.cc:304] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.208295 18785 raft_consensus.cc:2804] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.199880 18781 leader_election.cc:290] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.208678 18785 raft_consensus.cc:493] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.209095 18785 raft_consensus.cc:3060] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.210793 18194 raft_consensus.cc:2468] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:17:57.212353 18015 leader_election.cc:304] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.213402 18781 raft_consensus.cc:2804] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.213904 18781 raft_consensus.cc:697] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:57.214758 18781 consensus_queue.cc:237] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.231738 18665 raft_consensus.cc:493] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.232338 18665 raft_consensus.cc:515] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.235080 18665 leader_election.cc:290] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.236282 18194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
W20260709 13:17:57.241115 18017 leader_election.cc:336] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.241732 18017 leader_election.cc:336] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.238305 18782 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.252589 18123 raft_consensus.cc:2393] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 2: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:57.255487 18018 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.256567 18665 raft_consensus.cc:2749] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:17:57.268726 18091 leader_election.cc:336] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.274850 18782 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:57.275684 18017 leader_election.cc:336] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.276039 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:57.277066 18049 raft_consensus.cc:2393] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1787628aa7f6489497bb871588797230 in current term 2: Already voted for candidate fa03f4668c724e22a467d4e621f95272 in this term.
I20260709 13:17:57.278419 18090 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:57.283812 18785 raft_consensus.cc:515] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.288018 18784 raft_consensus.cc:493] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.288786 18784 raft_consensus.cc:515] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:57.292172 18782 raft_consensus.cc:2749] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.292737 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:57.292974 18782 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.293450 18782 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.295789 18782 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.303355 18784 leader_election.cc:290] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:57.305907 18091 leader_election.cc:336] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.306227 18194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.310631 18791 raft_consensus.cc:493] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.311292 18791 raft_consensus.cc:515] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:57.313242 18089 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:57.314905 18782 raft_consensus.cc:2749] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:57.317687 18165 leader_election.cc:336] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.318360 18165 leader_election.cc:304] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc8a9ec31a5745d69dae4262b271e568; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.319375 18782 raft_consensus.cc:493] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.320051 18782 raft_consensus.cc:515] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.321870 18784 raft_consensus.cc:2749] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.324255 18791 leader_election.cc:290] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:57.326362 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80b08582e1a4e099492c0650e820c45" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:57.326996 18049 raft_consensus.cc:3060] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.323145 18782 leader_election.cc:290] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.329909 18785 leader_election.cc:290] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:57.331563 18017 leader_election.cc:336] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.332082 18017 leader_election.cc:304] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:57.332893 18781 raft_consensus.cc:2749] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.333472 17871 catalog_manager.cc:5697] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.336045 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7187dfc4efc34078a81a1f59637521c8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.336687 18196 raft_consensus.cc:2468] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:57.339586 18781 raft_consensus.cc:493] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.341114 18781 raft_consensus.cc:515] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.334228 18049 raft_consensus.cc:2468] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:57.344520 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.345230 18089 leader_election.cc:304] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.346480 18782 raft_consensus.cc:2804] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.346587 18164 leader_election.cc:304] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.346880 18782 raft_consensus.cc:493] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.347594 18785 raft_consensus.cc:2804] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.350481 18785 raft_consensus.cc:697] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:57.343154 18781 leader_election.cc:290] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:57.351274 18165 leader_election.cc:336] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.353099 18091 leader_election.cc:336] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.353385 18784 raft_consensus.cc:493] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
W20260709 13:17:57.353642 18091 leader_election.cc:336] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.354066 18784 raft_consensus.cc:515] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.356570 18784 leader_election.cc:290] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.357501 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:57.357745 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.358181 18121 raft_consensus.cc:2468] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:57.358397 18196 raft_consensus.cc:2468] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:57.352121 18785 consensus_queue.cc:237] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.361019 18166 leader_election.cc:304] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.362205 18776 raft_consensus.cc:2804] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.362561 18776 raft_consensus.cc:493] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.362902 18776 raft_consensus.cc:3060] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.347523 18782 raft_consensus.cc:3060] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.365463 18791 raft_consensus.cc:493] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.366091 18784 raft_consensus.cc:493] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.366576 18784 raft_consensus.cc:515] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.367453 18089 leader_election.cc:304] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.368356 18792 raft_consensus.cc:2804] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.368709 18792 raft_consensus.cc:493] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.369027 18792 raft_consensus.cc:3060] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.370182 18782 raft_consensus.cc:515] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.376619 18791 raft_consensus.cc:515] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.379802 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:57.381405 18733 raft_consensus.cc:493] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.381953 18733 raft_consensus.cc:515] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.386942 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568" is_pre_election: true
I20260709 13:17:57.387692 18196 raft_consensus.cc:2468] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:57.389441 18015 leader_election.cc:304] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.390292 18665 raft_consensus.cc:2804] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.390643 18665 raft_consensus.cc:493] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.390972 18665 raft_consensus.cc:3060] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.385421 18733 leader_election.cc:290] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.381417 18782 leader_election.cc:290] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:57.399580 18091 leader_election.cc:336] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.400045 18091 leader_election.cc:304] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1787628aa7f6489497bb871588797230; no voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272
I20260709 13:17:57.400729 18779 raft_consensus.cc:2749] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.400568 18792 raft_consensus.cc:515] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:57.402230 18536 raft_consensus.cc:493] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.402855 18792 leader_election.cc:290] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:17:57.402745 18536 raft_consensus.cc:515] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.404672 18776 raft_consensus.cc:515] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.406726 18776 leader_election.cc:290] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.393226 18791 leader_election.cc:290] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.402776 18665 raft_consensus.cc:515] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
W20260709 13:17:57.425091 18091 leader_election.cc:336] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.426333 18017 leader_election.cc:336] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.426932 18017 leader_election.cc:304] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568
I20260709 13:17:57.457194 18784 leader_election.cc:290] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.459698 18665 leader_election.cc:290] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.462378 18536 leader_election.cc:290] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.463645 18733 raft_consensus.cc:2749] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:57.468654 18017 leader_election.cc:336] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.469132 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:57.469924 18121 raft_consensus.cc:2468] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:57.471217 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9de6c54e6314854955dc74a83ea79d3" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:57.471931 18196 raft_consensus.cc:3060] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:57.476153 18165 leader_election.cc:336] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.479043 17871 catalog_manager.cc:5697] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.481964 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7187dfc4efc34078a81a1f59637521c8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:57.482604 18198 raft_consensus.cc:3060] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.480127 18196 raft_consensus.cc:2468] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:17:57.487761 18015 leader_election.cc:304] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.489305 18784 raft_consensus.cc:493] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.490216 18198 raft_consensus.cc:2468] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:17:57.489976 18784 raft_consensus.cc:515] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.492316 18784 leader_election.cc:290] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.494133 18781 raft_consensus.cc:2804] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.495754 18166 leader_election.cc:304] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.495934 18194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00cc4ceb4dee449a8998229f3bd78288" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:17:57.496519 18194 raft_consensus.cc:3060] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.502005 18791 raft_consensus.cc:493] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.503854 18194 raft_consensus.cc:2468] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:17:57.504112 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c143c20585e74a46807a580f6dcdafc1" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:57.504716 18121 raft_consensus.cc:3060] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:57.505576 18165 leader_election.cc:336] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.506255 18165 leader_election.cc:336] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.509233 18017 leader_election.cc:336] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.510717 18781 raft_consensus.cc:697] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:57.511780 18781 consensus_queue.cc:237] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.512329 18121 raft_consensus.cc:2468] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:57.512758 18089 leader_election.cc:304] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.513804 18784 raft_consensus.cc:2804] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.514237 18784 raft_consensus.cc:493] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.515277 18784 raft_consensus.cc:3060] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.516714 18089 leader_election.cc:304] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.517553 18779 raft_consensus.cc:2804] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.518049 18779 raft_consensus.cc:697] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:57.518849 18779 consensus_queue.cc:237] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:17:57.514889 18792 raft_consensus.cc:2804] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:17:57.523676 18165 leader_election.cc:336] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.524140 18792 raft_consensus.cc:697] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:57.525118 18792 consensus_queue.cc:237] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:17:57.529647 18165 leader_election.cc:336] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.502642 18791 raft_consensus.cc:515] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.531682 18791 leader_election.cc:290] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.533921 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:57.534584 18050 raft_consensus.cc:2468] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:57.534817 18796 raft_consensus.cc:493] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.535516 18796 raft_consensus.cc:515] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.537163 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:57.537933 18050 raft_consensus.cc:2468] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 1.
I20260709 13:17:57.537382 17871 catalog_manager.cc:5697] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.541625 18796 leader_election.cc:290] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
W20260709 13:17:57.543494 18017 leader_election.cc:336] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.544131 18165 consensus_peers.cc:597] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.546614 18685 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.547103 18685 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.552717 18685 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
W20260709 13:17:57.554352 18091 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 93451.5us
W20260709 13:17:57.557830 18091 leader_election.cc:336] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.560700 18164 leader_election.cc:304] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.564990 18685 raft_consensus.cc:2804] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.565413 18685 raft_consensus.cc:493] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.565785 18685 raft_consensus.cc:3060] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:57.571137 18165 leader_election.cc:336] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.572778 18784 raft_consensus.cc:515] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.529721 18166 leader_election.cc:304] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.576515 18784 leader_election.cc:290] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.576778 18536 raft_consensus.cc:2804] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.577297 18536 raft_consensus.cc:697] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:57.578060 18536 consensus_queue.cc:237] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:17:57.587376 18165 leader_election.cc:336] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.590194 18091 leader_election.cc:336] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.591241 17871 catalog_manager.cc:5697] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.598570 18164 leader_election.cc:304] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.601080 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:57.602695 18796 raft_consensus.cc:493] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.603667 18796 raft_consensus.cc:515] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.605715 18796 leader_election.cc:290] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
W20260709 13:17:57.587741 18166 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.67:43399, user_credentials={real_user=slave}} blocked reactor thread for 58109.2us
I20260709 13:17:57.610414 18121 raft_consensus.cc:1275] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.611681 18048 raft_consensus.cc:2468] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 1.
I20260709 13:17:57.613921 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:57.614526 18121 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 3.
I20260709 13:17:57.615666 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc6bc2204cfe4f09a8f0bae7240822d8" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:57.615773 18090 leader_election.cc:304] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
W20260709 13:17:57.616183 18017 leader_election.cc:336] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.616792 18791 raft_consensus.cc:2804] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.617214 18776 raft_consensus.cc:2804] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.617197 18791 raft_consensus.cc:493] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.617759 18776 raft_consensus.cc:493] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.618221 18791 raft_consensus.cc:3060] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.618302 18776 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.617357 18121 raft_consensus.cc:3060] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.623070 18194 raft_consensus.cc:1275] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:17:57.629961 18089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 68228.7us
I20260709 13:17:57.630465 18796 consensus_queue.cc:1048] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.633822 18785 consensus_queue.cc:1048] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.635639 18166 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.638132 18121 raft_consensus.cc:2468] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:57.638365 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:57.638621 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230" is_pre_election: true
I20260709 13:17:57.639026 18784 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:57.639066 18123 raft_consensus.cc:2468] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 1.
I20260709 13:17:57.639657 18784 raft_consensus.cc:493] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.640038 18784 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:57.641953 18166 leader_election.cc:304] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.642462 18018 leader_election.cc:304] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.643003 18536 raft_consensus.cc:2804] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.642736 18791 raft_consensus.cc:515] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.649861 18784 raft_consensus.cc:515] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.650841 18124 raft_consensus.cc:2393] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa03f4668c724e22a467d4e621f95272 in current term 2: Already voted for candidate 1787628aa7f6489497bb871588797230 in this term.
I20260709 13:17:57.651729 18784 leader_election.cc:290] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:17:57.655200 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d0d9f275f44ae59f4f6c2df48acd6d" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:57.656157 18018 leader_election.cc:304] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa03f4668c724e22a467d4e621f95272; no voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.656481 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72bffd5492454e96ae4e10907b97381c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:57.656024 17871 catalog_manager.cc:5697] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.657099 18123 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:57.657336 18791 leader_election.cc:290] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.656038 18048 raft_consensus.cc:3060] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.643457 18685 raft_consensus.cc:515] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.667893 18685 leader_election.cc:290] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:57.673959 18017 consensus_peers.cc:597] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.675554 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6de10a3b7bb4a8da2902e5f42108ed2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:57.676168 18052 raft_consensus.cc:3060] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:57.676265 18091 leader_election.cc:336] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.678043 18776 raft_consensus.cc:515] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.682580 18048 raft_consensus.cc:2468] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:17:57.683211 18665 raft_consensus.cc:2749] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:57.686115 18090 leader_election.cc:304] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.687484 18799 raft_consensus.cc:2804] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.689347 18052 raft_consensus.cc:2468] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:57.690922 18164 leader_election.cc:304] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.691995 18799 raft_consensus.cc:697] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:57.693912 18799 consensus_queue.cc:237] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.657953 18796 raft_consensus.cc:2804] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:57.695842 18776 leader_election.cc:290] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:57.700084 18536 raft_consensus.cc:697] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:57.699826 18685 raft_consensus.cc:2804] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.696625 18123 raft_consensus.cc:2468] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 4.
I20260709 13:17:57.701555 18685 raft_consensus.cc:697] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:57.699491 18796 raft_consensus.cc:493] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 242d2387755e4a37ade4260e80f4eb6a)
I20260709 13:17:57.702502 18685 consensus_queue.cc:237] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.703020 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:57.703593 18052 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.703716 18166 leader_election.cc:304] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:17:57.708559 18776 raft_consensus.cc:2804] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:57.709059 18776 raft_consensus.cc:697] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 4 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:57.709848 18776 consensus_queue.cc:237] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.714145 18536 consensus_queue.cc:237] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:17:57.702502 18796 raft_consensus.cc:3060] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.726830 18052 raft_consensus.cc:2468] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:17:57.729419 18164 leader_election.cc:304] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
W20260709 13:17:57.730086 18165 leader_election.cc:336] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.730907 18165 leader_election.cc:336] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
W20260709 13:17:57.733783 18165 leader_election.cc:336] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.745002 18805 raft_consensus.cc:2804] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.770116 18805 raft_consensus.cc:697] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:17:57.771075 18805 consensus_queue.cc:237] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:17:57.810587 18796 raft_consensus.cc:515] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
W20260709 13:17:57.824353 18017 leader_election.cc:336] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:57.826529 18796 leader_election.cc:290] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:17:57.828985 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27dcd1944f074b89a6dd10d75e8c6dd2" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:17:57.829792 18121 raft_consensus.cc:3060] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.856024 17871 catalog_manager.cc:5697] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.857841 17871 catalog_manager.cc:5697] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.868227 18121 raft_consensus.cc:2468] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:17:57.879267 18018 leader_election.cc:304] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:57.880373 18796 raft_consensus.cc:2804] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.880861 18796 raft_consensus.cc:697] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:17:57.881623 18796 consensus_queue.cc:237] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:57.886023 17871 catalog_manager.cc:5697] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.887691 17871 catalog_manager.cc:5697] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.891155 17871 catalog_manager.cc:5697] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 3 to 4, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.897954 18052 raft_consensus.cc:1275] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.899556 18776 consensus_queue.cc:1048] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:57.912495 18165 consensus_peers.cc:597] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:57.914634 18017 consensus_peers.cc:597] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.929816 17871 catalog_manager.cc:5697] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.952865 17871 catalog_manager.cc:5697] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 1 to 2, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.962206 18196 raft_consensus.cc:1275] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 1787628aa7f6489497bb871588797230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.965888 18121 raft_consensus.cc:1275] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.965790 18791 consensus_queue.cc:1048] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.967361 18805 consensus_queue.cc:1048] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:57.997366 18165 consensus_peers.cc:597] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:58.000211 18091 consensus_peers.cc:597] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:58.031323 18091 consensus_peers.cc:597] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.062327 18196 raft_consensus.cc:1275] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.063845 18796 consensus_queue.cc:1048] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.102928 18052 raft_consensus.cc:1275] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.105232 18813 consensus_queue.cc:1048] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:58.114086 18165 consensus_peers.cc:597] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.123064 18091 consensus_peers.cc:597] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:58.156390 18196 raft_consensus.cc:1275] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 1787628aa7f6489497bb871588797230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:17:58.156890 18165 consensus_peers.cc:597] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.158150 18791 consensus_queue.cc:1048] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:58.173502 18017 consensus_peers.cc:597] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.180050 18091 consensus_peers.cc:597] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.188519 18165 consensus_peers.cc:597] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.191898 18052 raft_consensus.cc:1275] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 1787628aa7f6489497bb871588797230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.193640 18791 consensus_queue.cc:1048] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:58.197541 10257 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 17998)
I20260709 13:17:58.237505 18121 raft_consensus.cc:1275] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 4 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:58.239230 18784 consensus_queue.cc:1048] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:17:58.245749 18165 consensus_peers.cc:597] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:58.248739 18091 consensus_peers.cc:597] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:58.414067 18165 consensus_peers.cc:597] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.419682 18017 consensus_peers.cc:597] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:58.420255 18017 consensus_peers.cc:597] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.420485 18165 consensus_peers.cc:597] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.423501 18017 consensus_peers.cc:597] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.423458 18165 consensus_peers.cc:597] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.429453 18052 raft_consensus.cc:1275] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.431594 18124 raft_consensus.cc:1275] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:17:58.435488 18091 consensus_peers.cc:597] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.439944 18091 consensus_peers.cc:597] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.439343 18165 consensus_peers.cc:597] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:58.458566 18776 consensus_queue.cc:1048] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:58.461043 18089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 60784.1us
I20260709 13:17:58.469794 18124 raft_consensus.cc:1275] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.482471 18805 consensus_queue.cc:1048] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.498539 18733 consensus_queue.cc:1048] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:58.519909 18165 consensus_peers.cc:597] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.540736 18017 consensus_peers.cc:597] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.549302 18091 consensus_peers.cc:597] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.550238 18017 consensus_peers.cc:597] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.560096 18165 consensus_peers.cc:597] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.560881 18091 consensus_peers.cc:597] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.561533 18091 consensus_peers.cc:597] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.578181 18165 consensus_peers.cc:597] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.584682 18091 consensus_peers.cc:597] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.593051 18091 consensus_peers.cc:597] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.593091 18017 consensus_peers.cc:597] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.634840 18091 consensus_peers.cc:597] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.692154 18091 consensus_peers.cc:597] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.704861 18165 consensus_peers.cc:597] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.745262 18091 consensus_peers.cc:597] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.747588 18091 consensus_peers.cc:597] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.755505 18091 consensus_peers.cc:597] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.812142 18017 consensus_peers.cc:597] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:58.944689 18165 consensus_peers.cc:597] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:59.022567 18091 consensus_peers.cc:597] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:59.198088 10257 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 17998)
I20260709 13:17:59.312748 18838 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.315714 18838 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:59.328274 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272" is_pre_election: true
I20260709 13:17:59.328958 18048 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:17:59.329002 18838 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:17:59.332083 18090 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:17:59.333765 18839 raft_consensus.cc:2804] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:59.334201 18839 raft_consensus.cc:493] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.339807 18839 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:59.338056 18091 leader_election.cc:336] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:59.348028 18839 raft_consensus.cc:515] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:59.354295 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0044e0ba4e7548ea8cd3452eed875fa8" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:17:59.354861 18048 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:59.356945 18839 leader_election.cc:290] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 3 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
W20260709 13:17:59.358618 18091 leader_election.cc:336] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111)
I20260709 13:17:59.361281 18048 raft_consensus.cc:2468] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 3.
I20260709 13:17:59.362404 18090 leader_election.cc:304] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:17:59.363441 18839 raft_consensus.cc:2804] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:59.364008 18839 raft_consensus.cc:697] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 3 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:17:59.364989 18839 consensus_queue.cc:237] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:17:59.382779 17875 catalog_manager.cc:5697] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 1 to 3, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:59.801312 18091 consensus_peers.cc:597] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:59.930433 18048 raft_consensus.cc:1275] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Refusing update from remote peer 1787628aa7f6489497bb871588797230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:59.936336 18829 consensus_queue.cc:1048] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:17:59.936772 18165 consensus_peers.cc:597] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.128393 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 178031us
W20260709 13:18:00.198590 10257 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 17998)
W20260709 13:18:00.234850 18017 consensus_peers.cc:597] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.447674 18091 consensus_peers.cc:597] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.490046 18091 consensus_peers.cc:597] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.517047 18165 consensus_peers.cc:597] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.523300 18165 consensus_peers.cc:597] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.526556 18091 consensus_peers.cc:597] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.547369 18165 consensus_peers.cc:597] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.550906 18017 consensus_peers.cc:597] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.559984 18165 consensus_peers.cc:597] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.634469 18017 consensus_peers.cc:597] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.678371 18165 consensus_peers.cc:597] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.775059 18165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111) [suppressed 429 similar messages]
W20260709 13:18:00.780580 18165 consensus_peers.cc:597] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.781128 18091 consensus_peers.cc:597] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.796375 18165 consensus_peers.cc:597] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.815259 18165 consensus_peers.cc:597] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.817037 18091 consensus_peers.cc:597] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:00.828711 18165 consensus_peers.cc:597] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.852360 18165 consensus_peers.cc:597] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.854694 18017 consensus_peers.cc:597] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.898522 18165 consensus_peers.cc:597] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:00.987344 18017 consensus_peers.cc:597] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:01.005709 18091 consensus_peers.cc:597] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.006345 18091 consensus_peers.cc:597] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.045238 18091 consensus_peers.cc:597] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.104522 18017 consensus_peers.cc:597] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.112730 18091 consensus_peers.cc:597] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.113384 18091 consensus_peers.cc:597] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.120676 18165 consensus_peers.cc:597] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.124589 18017 consensus_peers.cc:597] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.125523 18091 consensus_peers.cc:597] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.126099 18091 consensus_peers.cc:597] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.132314 18091 consensus_peers.cc:597] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.138487 18017 consensus_peers.cc:597] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.181216 18165 consensus_peers.cc:597] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.188645 18091 consensus_peers.cc:597] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.199155 10257 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 17998)
W20260709 13:18:01.215094 18091 consensus_peers.cc:597] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.236984 18091 consensus_peers.cc:597] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.308753 18017 consensus_peers.cc:597] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.335760 18165 consensus_peers.cc:597] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.363366 18091 consensus_peers.cc:597] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.408151 18017 consensus_peers.cc:597] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.432865 18091 consensus_peers.cc:597] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:01.493819 18165 consensus_peers.cc:597] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:02.200132 10257 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 17998)
W20260709 13:18:02.216138 18091 consensus_peers.cc:597] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:02.256750 18165 consensus_peers.cc:597] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:02.779320 18165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 44669.5us
I20260709 13:18:02.800005 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
W20260709 13:18:02.823045 18165 consensus_peers.cc:597] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:02.857465 18091 consensus_peers.cc:597] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:02.869688 18017 consensus_peers.cc:597] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:18:02.907673 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:02.916502 18865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:02.919186 18864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:02.923724 10257 server_base.cc:1061] running on GCE node
W20260709 13:18:02.930783 18017 consensus_peers.cc:597] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:02.930946 18165 consensus_peers.cc:597] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:02.930421 18867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:02.934695 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:02.935094 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:02.935343 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603082935321 us; error 0 us; skew 500 ppm
I20260709 13:18:02.935966 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:02.939342 10257 webserver.cc:533] Webserver started at http://127.10.4.65:33245/ using document root <none> and password file <none>
I20260709 13:18:02.940096 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:02.940409 10257 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:18:02.948212 18017 consensus_peers.cc:597] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:02.952991 10257 fs_manager.cc:714] Time spent opening directory manager: real 0.011s	user 0.005s	sys 0.003s
W20260709 13:18:02.962347 18091 consensus_peers.cc:597] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:18:02.963481 18872 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:02.964613 10257 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:18:02.965003 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "242d2387755e4a37ade4260e80f4eb6a"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-gkm9"
I20260709 13:18:02.965512 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:18:03.084126 18091 consensus_peers.cc:597] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.096823 18165 consensus_peers.cc:597] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:18:03.107036 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:03.111281 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:03.113785 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:03.119573 18880 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:18:03.152117 18165 consensus_peers.cc:597] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.189678 18165 consensus_peers.cc:597] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:03.228855 18091 consensus_peers.cc:597] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.231366 18165 consensus_peers.cc:597] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.246477 18017 consensus_peers.cc:597] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.288975 18165 consensus_peers.cc:597] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:03.351501 18165 consensus_peers.cc:597] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.360333 18165 consensus_peers.cc:597] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.368340 18091 consensus_peers.cc:597] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:03.403637 18091 consensus_peers.cc:597] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.442499 18091 consensus_peers.cc:597] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.466213 18091 consensus_peers.cc:597] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.521633 18091 consensus_peers.cc:597] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.530577 18091 consensus_peers.cc:597] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.535615 18165 consensus_peers.cc:597] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.537945 18091 consensus_peers.cc:597] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.560902 18017 consensus_peers.cc:597] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:03.570698 18165 consensus_peers.cc:597] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:03.590642 18165 consensus_peers.cc:597] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.616185 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:18:03.616465 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.498s	user 0.015s	sys 0.000s
I20260709 13:18:03.616752 10257 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:18:03.623279 18880 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:18:03.644037 18017 consensus_peers.cc:597] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.660782 18880 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.661885 18880 tablet_bootstrap.cc:492] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.662573 18880 ts_tablet_manager.cc:1403] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.017s
I20260709 13:18:03.665266 18880 raft_consensus.cc:359] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.666405 18880 raft_consensus.cc:740] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.667200 18880 consensus_queue.cc:260] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.671931 18880 ts_tablet_manager.cc:1434] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.006s
I20260709 13:18:03.672747 18880 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:03.686594 18880 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.687635 18880 tablet_bootstrap.cc:492] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.688297 18880 ts_tablet_manager.cc:1403] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:18:03.690852 18880 raft_consensus.cc:359] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.691550 18880 raft_consensus.cc:740] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.692358 18880 consensus_queue.cc:260] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.694372 18880 ts_tablet_manager.cc:1434] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 13:18:03.695194 18880 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:03.710166 18880 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.711066 18880 tablet_bootstrap.cc:492] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.711687 18880 ts_tablet_manager.cc:1403] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:18:03.713979 18880 raft_consensus.cc:359] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:18:03.714569 18017 consensus_peers.cc:597] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.714645 18880 raft_consensus.cc:740] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.715839 18880 consensus_queue.cc:260] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.718003 18880 ts_tablet_manager.cc:1434] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:03.718851 18880 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:18:03.719313 18091 consensus_peers.cc:597] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.731812 18091 consensus_peers.cc:597] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.735623 18880 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.736657 18880 tablet_bootstrap.cc:492] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.737334 18880 ts_tablet_manager.cc:1403] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:18:03.739869 18880 raft_consensus.cc:359] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.740538 18880 raft_consensus.cc:740] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.741207 18880 consensus_queue.cc:260] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.743117 18880 ts_tablet_manager.cc:1434] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
W20260709 13:18:03.743036 18091 consensus_peers.cc:597] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.744124 18880 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:18:03.762111 18091 consensus_peers.cc:597] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.772552 18880 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.773795 18880 tablet_bootstrap.cc:492] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.774475 18880 ts_tablet_manager.cc:1403] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.012s
W20260709 13:18:03.774397 18017 consensus_peers.cc:597] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.777382 18880 raft_consensus.cc:359] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:03.778170 18880 raft_consensus.cc:740] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.779117 18880 consensus_queue.cc:260] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:03.780756 18880 ts_tablet_manager.cc:1434] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:03.781440 18880 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:03.797679 18880 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.798769 18880 tablet_bootstrap.cc:492] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.799456 18880 ts_tablet_manager.cc:1403] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 13:18:03.801990 18880 raft_consensus.cc:359] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:03.802673 18880 raft_consensus.cc:740] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.803388 18880 consensus_queue.cc:260] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:03.806025 18880 ts_tablet_manager.cc:1434] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:03.806846 18880 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:18:03.811499 18091 consensus_peers.cc:597] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.821525 18880 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.822412 18880 tablet_bootstrap.cc:492] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.823019 18880 ts_tablet_manager.cc:1403] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
W20260709 13:18:03.825119 18091 consensus_peers.cc:597] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.825522 18880 raft_consensus.cc:359] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.826197 18880 raft_consensus.cc:740] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.826814 18880 consensus_queue.cc:260] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.828244 18880 ts_tablet_manager.cc:1434] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:18:03.828804 18880 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:03.840723 18880 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.841706 18880 tablet_bootstrap.cc:492] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.842355 18880 ts_tablet_manager.cc:1403] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.002s
I20260709 13:18:03.846954 18880 raft_consensus.cc:359] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.847479 18880 raft_consensus.cc:740] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.848039 18880 consensus_queue.cc:260] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.849541 18880 ts_tablet_manager.cc:1434] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:18:03.850126 18880 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:03.864424 18880 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.865345 18880 tablet_bootstrap.cc:492] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.865978 18880 ts_tablet_manager.cc:1403] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.002s
I20260709 13:18:03.868889 18880 raft_consensus.cc:359] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:03.869524 18880 raft_consensus.cc:740] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.870289 18880 consensus_queue.cc:260] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:03.873217 10257 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:18:03.873595 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.257s	user 0.235s	sys 0.013s
I20260709 13:18:03.873238 18880 ts_tablet_manager.cc:1434] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:03.875742 18880 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:18:03.878216 18017 consensus_peers.cc:597] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.883803 18017 consensus_peers.cc:597] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:18:03.887959 18165 consensus_peers.cc:597] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.893115 18880 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.894701 18880 tablet_bootstrap.cc:492] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.895375 18880 ts_tablet_manager.cc:1403] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.000s
W20260709 13:18:03.896693 18165 consensus_peers.cc:597] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.898016 18880 raft_consensus.cc:359] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.899914 18880 raft_consensus.cc:740] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.901015 18880 consensus_queue.cc:260] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:03.903079 18880 ts_tablet_manager.cc:1434] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:18:03.903870 18880 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:18:03.923475 18165 consensus_peers.cc:597] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.931123 18880 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.933991 18880 tablet_bootstrap.cc:492] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.934638 18880 ts_tablet_manager.cc:1403] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.001s
I20260709 13:18:03.937333 18880 raft_consensus.cc:359] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
W20260709 13:18:03.940183 18091 consensus_peers.cc:597] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:18:03.940536 18880 raft_consensus.cc:740] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.941440 18880 consensus_queue.cc:260] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.943977 18880 ts_tablet_manager.cc:1434] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:18:03.944859 18880 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:03.957901 18880 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.958904 18880 tablet_bootstrap.cc:492] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.960493 18880 ts_tablet_manager.cc:1403] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:18:03.964519 18880 raft_consensus.cc:359] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.965059 18880 raft_consensus.cc:740] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.965579 18880 consensus_queue.cc:260] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.967062 18880 ts_tablet_manager.cc:1434] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:18:03.967720 18880 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:03.981266 18880 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.982159 18880 tablet_bootstrap.cc:492] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:03.982666 18880 ts_tablet_manager.cc:1403] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:18:03.984701 18880 raft_consensus.cc:359] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.985335 18880 raft_consensus.cc:740] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.986271 18880 consensus_queue.cc:260] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:03.987813 18880 ts_tablet_manager.cc:1434] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:18:03.988423 18880 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.011385 18880 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.012450 18880 tablet_bootstrap.cc:492] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.013159 18880 ts_tablet_manager.cc:1403] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.008s
I20260709 13:18:04.018846 18880 raft_consensus.cc:359] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.019237 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:32893
I20260709 13:18:04.019490 18954 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:32893 every 8 connection(s)
I20260709 13:18:04.019646 18880 raft_consensus.cc:740] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.020702 18880 consensus_queue.cc:260] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.022581 18880 ts_tablet_manager.cc:1434] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:18:04.023838 18880 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.066480 18880 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.067385 18880 tablet_bootstrap.cc:492] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.073905 18880 ts_tablet_manager.cc:1403] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.050s	user 0.030s	sys 0.004s
I20260709 13:18:04.076289 18929 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.077497 18880 raft_consensus.cc:359] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.078226 18880 raft_consensus.cc:740] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.078892 18880 consensus_queue.cc:260] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.081089 18880 ts_tablet_manager.cc:1434] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:04.081939 18880 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.093173 18956 heartbeater.cc:344] Connected to a master server at 127.10.4.126:39849
I20260709 13:18:04.093684 18956 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:04.110770 18880 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.111876 18880 tablet_bootstrap.cc:492] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.112612 18880 ts_tablet_manager.cc:1403] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.009s
I20260709 13:18:04.094609 18956 heartbeater.cc:507] Master 127.10.4.126:39849 requested a full tablet report, sending...
I20260709 13:18:04.115660 18880 raft_consensus.cc:359] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.116539 18880 raft_consensus.cc:740] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.117447 18880 consensus_queue.cc:260] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.120189 18880 ts_tablet_manager.cc:1434] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:18:04.121713 18880 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.140446 18880 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.142200 18880 tablet_bootstrap.cc:492] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.142791 18880 ts_tablet_manager.cc:1403] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:18:04.145094 18880 raft_consensus.cc:359] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.145649 18880 raft_consensus.cc:740] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.146188 18880 consensus_queue.cc:260] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.148146 18880 ts_tablet_manager.cc:1434] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:04.148627 17872 ts_manager.cc:194] Re-registered known tserver with Master: 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893)
I20260709 13:18:04.148998 18880 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.149539 18929 raft_consensus.cc:3060] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.169997 18926 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.172185 18880 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.173214 18880 tablet_bootstrap.cc:492] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.173895 18880 ts_tablet_manager.cc:1403] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.011s
I20260709 13:18:04.178514 18880 raft_consensus.cc:359] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.179572 18880 raft_consensus.cc:740] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.180558 18880 consensus_queue.cc:260] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.182806 18880 ts_tablet_manager.cc:1434] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:18:04.183796 18880 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.208500 18880 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.209393 18880 tablet_bootstrap.cc:492] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.210008 18880 ts_tablet_manager.cc:1403] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.005s
I20260709 13:18:04.210405 17872 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53290
I20260709 13:18:04.212512 18880 raft_consensus.cc:359] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.213120 18880 raft_consensus.cc:740] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.213640 18880 consensus_queue.cc:260] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.215065 18880 ts_tablet_manager.cc:1434] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:18:04.215543 18956 heartbeater.cc:499] Master 127.10.4.126:39849 was elected leader, sending a full tablet report...
I20260709 13:18:04.215785 18880 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.228948 18880 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.229866 18880 tablet_bootstrap.cc:492] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.230454 18880 ts_tablet_manager.cc:1403] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:18:04.232419 18880 raft_consensus.cc:359] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.232931 18880 raft_consensus.cc:740] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.233494 18880 consensus_queue.cc:260] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.235191 18880 ts_tablet_manager.cc:1434] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 13:18:04.235976 18880 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.250326 18880 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.251223 18880 tablet_bootstrap.cc:492] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.251852 18880 ts_tablet_manager.cc:1403] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:18:04.254225 18880 raft_consensus.cc:359] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.254886 18880 raft_consensus.cc:740] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.255573 18880 consensus_queue.cc:260] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.257614 18880 ts_tablet_manager.cc:1434] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:18:04.258491 18880 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.273253 18880 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.274144 18880 tablet_bootstrap.cc:492] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.274766 18880 ts_tablet_manager.cc:1403] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.005s
I20260709 13:18:04.277211 18880 raft_consensus.cc:359] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.277842 18880 raft_consensus.cc:740] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.278506 18880 consensus_queue.cc:260] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.280462 18880 ts_tablet_manager.cc:1434] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.002s
I20260709 13:18:04.281082 18880 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.294416 18880 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.295302 18880 tablet_bootstrap.cc:492] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.295953 18880 ts_tablet_manager.cc:1403] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:18:04.297863 18880 raft_consensus.cc:359] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.298312 18880 raft_consensus.cc:740] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.298851 18880 consensus_queue.cc:260] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.300446 18880 ts_tablet_manager.cc:1434] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:04.301007 18880 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.314716 18880 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.315672 18880 tablet_bootstrap.cc:492] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.316447 18880 ts_tablet_manager.cc:1403] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.001s
I20260709 13:18:04.320284 18880 raft_consensus.cc:359] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.320952 18880 raft_consensus.cc:740] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.321581 18880 consensus_queue.cc:260] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.323243 18880 ts_tablet_manager.cc:1434] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:04.324079 18880 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.331588 18929 raft_consensus.cc:3060] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.339648 18880 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.340726 18880 tablet_bootstrap.cc:492] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.341377 18880 ts_tablet_manager.cc:1403] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:18:04.343887 18880 raft_consensus.cc:359] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.344635 18880 raft_consensus.cc:740] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.345327 18880 consensus_queue.cc:260] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.347383 18880 ts_tablet_manager.cc:1434] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:04.348218 18880 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.356426 18926 raft_consensus.cc:3060] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.374739 18929 raft_consensus.cc:3060] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.383704 18880 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.384702 18880 tablet_bootstrap.cc:492] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.385385 18880 ts_tablet_manager.cc:1403] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.037s	user 0.003s	sys 0.020s
I20260709 13:18:04.388051 18880 raft_consensus.cc:359] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.388816 18880 raft_consensus.cc:740] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.389906 18880 consensus_queue.cc:260] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.392001 18880 ts_tablet_manager.cc:1434] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:04.392791 18880 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.408099 18880 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.409317 18880 tablet_bootstrap.cc:492] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.409997 18880 ts_tablet_manager.cc:1403] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.007s
I20260709 13:18:04.413516 18880 raft_consensus.cc:359] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.414461 18880 raft_consensus.cc:740] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.415347 18929 raft_consensus.cc:3060] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.415158 18880 consensus_queue.cc:260] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.417488 18880 ts_tablet_manager.cc:1434] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:04.426009 18880 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.435003 18926 raft_consensus.cc:3060] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.483799 18880 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.484848 18880 tablet_bootstrap.cc:492] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.485543 18880 ts_tablet_manager.cc:1403] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.060s	user 0.029s	sys 0.005s
I20260709 13:18:04.488121 18880 raft_consensus.cc:359] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.488811 18880 raft_consensus.cc:740] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.489512 18880 consensus_queue.cc:260] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.491641 18880 ts_tablet_manager.cc:1434] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:04.492453 18880 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.507236 18880 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.508069 18880 tablet_bootstrap.cc:492] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.509728 18880 ts_tablet_manager.cc:1403] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:18:04.512452 18880 raft_consensus.cc:359] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.513178 18880 raft_consensus.cc:740] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.513878 18880 consensus_queue.cc:260] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.516347 18880 ts_tablet_manager.cc:1434] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 13:18:04.517925 18880 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.529990 18880 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.530731 18880 tablet_bootstrap.cc:492] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.531350 18880 ts_tablet_manager.cc:1403] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:18:04.533345 18880 raft_consensus.cc:359] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:04.533831 18880 raft_consensus.cc:740] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.534334 18880 consensus_queue.cc:260] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:04.536167 18880 ts_tablet_manager.cc:1434] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:04.536895 18880 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.551733 18880 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.552510 18880 tablet_bootstrap.cc:492] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.553037 18880 ts_tablet_manager.cc:1403] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:18:04.554953 18880 raft_consensus.cc:359] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.555483 18880 raft_consensus.cc:740] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.556010 18880 consensus_queue.cc:260] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.557986 18880 ts_tablet_manager.cc:1434] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:04.559095 18880 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.572465 18880 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.573437 18880 tablet_bootstrap.cc:492] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.574082 18880 ts_tablet_manager.cc:1403] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:18:04.576138 18880 raft_consensus.cc:359] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.576606 18880 raft_consensus.cc:740] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.577966 18880 consensus_queue.cc:260] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.584313 18926 raft_consensus.cc:3060] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.593106 18880 ts_tablet_manager.cc:1434] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.019s	user 0.015s	sys 0.004s
I20260709 13:18:04.593990 18880 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.613828 18880 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.614566 18880 tablet_bootstrap.cc:492] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.615087 18880 ts_tablet_manager.cc:1403] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.001s
I20260709 13:18:04.617491 18880 raft_consensus.cc:359] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.618144 18880 raft_consensus.cc:740] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.618875 18880 consensus_queue.cc:260] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.621098 18880 ts_tablet_manager.cc:1434] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:18:04.621958 18880 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.638695 18880 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.639432 18880 tablet_bootstrap.cc:492] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.639957 18880 ts_tablet_manager.cc:1403] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:18:04.641891 18880 raft_consensus.cc:359] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.642518 18880 raft_consensus.cc:740] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.643010 18880 consensus_queue.cc:260] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.644553 18880 ts_tablet_manager.cc:1434] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:04.645416 18880 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.658622 18880 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.659551 18880 tablet_bootstrap.cc:492] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.660087 18880 ts_tablet_manager.cc:1403] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:18:04.662148 18880 raft_consensus.cc:359] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.662700 18880 raft_consensus.cc:740] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.663290 18880 consensus_queue.cc:260] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.665179 18880 ts_tablet_manager.cc:1434] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:18:04.666304 18880 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.687184 18880 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.687991 18880 tablet_bootstrap.cc:492] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.688632 18880 ts_tablet_manager.cc:1403] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 13:18:04.691099 18880 raft_consensus.cc:359] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.691824 18880 raft_consensus.cc:740] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.692495 18880 consensus_queue.cc:260] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.694785 18880 ts_tablet_manager.cc:1434] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:18:04.695654 18880 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.709652 18926 raft_consensus.cc:3060] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.714069 18880 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.719599 18880 tablet_bootstrap.cc:492] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.720269 18880 ts_tablet_manager.cc:1403] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.000s
I20260709 13:18:04.723093 18880 raft_consensus.cc:359] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.723769 18880 raft_consensus.cc:740] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.724393 18880 consensus_queue.cc:260] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.727411 18880 ts_tablet_manager.cc:1434] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:04.728896 18880 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.751703 18880 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.752676 18880 tablet_bootstrap.cc:492] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.753275 18880 ts_tablet_manager.cc:1403] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.003s
I20260709 13:18:04.755772 18880 raft_consensus.cc:359] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.756477 18880 raft_consensus.cc:740] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.757159 18880 consensus_queue.cc:260] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.761278 18880 ts_tablet_manager.cc:1434] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:18:04.767359 18880 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
W20260709 13:18:04.822683 18089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 90548.9us
W20260709 13:18:04.830610 18090 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 98873.2us
I20260709 13:18:04.832243 18880 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:18:04.831547 18091 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:32893, user_credentials={real_user=slave}} blocked reactor thread for 63892.3us
I20260709 13:18:04.833591 18880 tablet_bootstrap.cc:492] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.836236 18880 ts_tablet_manager.cc:1403] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.069s	user 0.030s	sys 0.029s
I20260709 13:18:04.838537 18926 raft_consensus.cc:3060] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.838829 18880 raft_consensus.cc:359] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.839612 18880 raft_consensus.cc:740] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.840227 18880 consensus_queue.cc:260] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.842300 18880 ts_tablet_manager.cc:1434] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:04.843055 18880 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.895522 18880 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.896489 18880 tablet_bootstrap.cc:492] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.897159 18880 ts_tablet_manager.cc:1403] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.054s	user 0.026s	sys 0.023s
I20260709 13:18:04.899822 18880 raft_consensus.cc:359] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.900501 18880 raft_consensus.cc:740] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.901293 18880 consensus_queue.cc:260] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:04.903419 18880 ts_tablet_manager.cc:1434] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:04.904649 18880 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.920147 18880 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.920872 18880 tablet_bootstrap.cc:492] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.921373 18880 ts_tablet_manager.cc:1403] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:18:04.923338 18880 raft_consensus.cc:359] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.924033 18880 raft_consensus.cc:740] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.925984 18880 consensus_queue.cc:260] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.928043 18880 ts_tablet_manager.cc:1434] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:18:04.930197 18880 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.958981 18880 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.959956 18880 tablet_bootstrap.cc:492] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.960618 18880 ts_tablet_manager.cc:1403] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.008s
I20260709 13:18:04.962890 18880 raft_consensus.cc:359] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.963537 18880 raft_consensus.cc:740] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.964241 18880 consensus_queue.cc:260] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:04.966260 18880 ts_tablet_manager.cc:1434] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:04.967363 18880 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:04.975263 18926 raft_consensus.cc:3060] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:04.982081 18880 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:04.983052 18880 tablet_bootstrap.cc:492] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:04.983724 18880 ts_tablet_manager.cc:1403] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 13:18:04.986106 18880 raft_consensus.cc:359] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.986750 18880 raft_consensus.cc:740] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.987541 18880 consensus_queue.cc:260] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:04.989832 18880 ts_tablet_manager.cc:1434] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:04.995761 18880 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:05.009508 18926 raft_consensus.cc:3060] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.010279 18880 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:05.010977 18880 tablet_bootstrap.cc:492] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:05.011504 18880 ts_tablet_manager.cc:1403] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:18:05.013276 18880 raft_consensus.cc:359] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:05.013728 18880 raft_consensus.cc:740] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.014233 18880 consensus_queue.cc:260] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:05.016419 18880 ts_tablet_manager.cc:1434] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:05.017298 18880 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap starting.
I20260709 13:18:05.034986 18880 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:05.035907 18880 tablet_bootstrap.cc:492] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Bootstrap complete.
I20260709 13:18:05.036537 18880 ts_tablet_manager.cc:1403] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:18:05.038772 18880 raft_consensus.cc:359] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:05.039425 18880 raft_consensus.cc:740] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Initialized, Role: FOLLOWER
I20260709 13:18:05.040086 18880 consensus_queue.cc:260] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:05.042126 18880 ts_tablet_manager.cc:1434] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:05.277477 18925 raft_consensus.cc:3060] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.055259 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:14.106343 18121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:14.107206 18121 raft_consensus.cc:606] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:14.121325 18200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:14.121918 18200 raft_consensus.cc:606] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:14.124578 18121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:14.125052 18121 raft_consensus.cc:606] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:14.127218 18121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:14.127707 18121 raft_consensus.cc:606] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:14.129725 18121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:14.130141 18121 raft_consensus.cc:606] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:14.133354 18121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:14.133890 18121 raft_consensus.cc:606] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS fa03f4668c724e22a467d4e621f95272
I20260709 13:18:14.134940 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:14.135326 18862 raft_consensus.cc:993] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.135908 19026 raft_consensus.cc:1081] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.136315 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:14.137423 18927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:18:14.137940 18927 raft_consensus.cc:493] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.138217 18927 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.144888 18927 raft_consensus.cc:515] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.147348 18927 leader_election.cc:290] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:14.166211 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:14.166851 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c18e4eea964029ad14dc02ce2137e2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:14.167501 18196 raft_consensus.cc:3055] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:14.167042 18121 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.167786 18196 raft_consensus.cc:740] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:14.168506 19031 raft_consensus.cc:993] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.169032 19031 raft_consensus.cc:1081] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.168733 18196 consensus_queue.cc:260] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.170173 18196 raft_consensus.cc:3060] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.171141 18926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:14.171739 18926 raft_consensus.cc:493] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.172065 18926 raft_consensus.cc:3060] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.176573 18121 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 3.
I20260709 13:18:14.177682 18877 leader_election.cc:304] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:18:14.177851 18196 raft_consensus.cc:2468] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 3.
I20260709 13:18:14.179453 19038 raft_consensus.cc:2804] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:14.179481 18926 raft_consensus.cc:515] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:14.180799 19038 raft_consensus.cc:697] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:14.181536 18926 leader_election.cc:290] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:14.181767 19038 consensus_queue.cc:237] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.183290 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:14.183854 18121 raft_consensus.cc:3055] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:14.184077 18121 raft_consensus.cc:740] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:14.184793 18121 consensus_queue.cc:260] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:14.185914 18121 raft_consensus.cc:3060] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.196188 18121 raft_consensus.cc:2468] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.196782 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8989b7b66db644f89f6c4421fd51f2cc" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:14.197458 18050 raft_consensus.cc:3060] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.197453 18877 leader_election.cc:304] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:18:14.198518 19038 raft_consensus.cc:2804] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:14.199499 19038 raft_consensus.cc:697] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:14.200448 19038 consensus_queue.cc:237] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:14.203687 17872 catalog_manager.cc:5697] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 2 to 3, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 3 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.207549 18050 raft_consensus.cc:2468] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.237129 17872 catalog_manager.cc:5697] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:14.415582 19022 raft_consensus.cc:993] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.416205 19025 raft_consensus.cc:1081] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.418466 18927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:14.419070 18927 raft_consensus.cc:493] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.419442 18927 raft_consensus.cc:3060] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.424559 18927 raft_consensus.cc:515] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.426271 18927 leader_election.cc:290] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:14.426858 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:14.427139 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c67ac9ecbb841cea07be431d6a78d54" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:14.427417 18121 raft_consensus.cc:3055] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:14.427707 18196 raft_consensus.cc:3060] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.427769 18121 raft_consensus.cc:740] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:14.428781 18121 consensus_queue.cc:260] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.429867 18121 raft_consensus.cc:3060] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.433203 18196 raft_consensus.cc:2468] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.434439 18873 leader_election.cc:304] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:14.435197 19038 raft_consensus.cc:2804] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:14.435585 19038 raft_consensus.cc:697] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:14.435787 18121 raft_consensus.cc:2468] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.436482 19038 consensus_queue.cc:237] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.445963 17872 catalog_manager.cc:5697] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.452773 19022 raft_consensus.cc:993] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.453276 19022 raft_consensus.cc:1081] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.454815 18926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:14.455251 18926 raft_consensus.cc:493] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.455544 18926 raft_consensus.cc:3060] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.461474 18926 raft_consensus.cc:515] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:14.463970 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:14.464638 18121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f4120b61f1d44f6a4a3c5c4575f993a" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:14.464768 18926 leader_election.cc:290] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:14.465320 18121 raft_consensus.cc:3055] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:14.464617 18050 raft_consensus.cc:3060] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.465890 18121 raft_consensus.cc:740] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:14.466813 18121 consensus_queue.cc:260] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:14.468115 18121 raft_consensus.cc:3060] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.474807 18121 raft_consensus.cc:2468] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.476176 18877 leader_election.cc:304] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:18:14.477101 19038 raft_consensus.cc:2804] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:14.477649 19038 raft_consensus.cc:697] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:14.477995 18050 raft_consensus.cc:2468] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.478592 19038 consensus_queue.cc:237] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:14.492805 17872 catalog_manager.cc:5697] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:14.540143 19031 raft_consensus.cc:993] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230: : Instructing follower fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:14.540643 19031 raft_consensus.cc:1081] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:14.543517 18048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f"
dest_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:44110
I20260709 13:18:14.544181 18048 raft_consensus.cc:493] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.544586 18048 raft_consensus.cc:3060] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.552863 18048 raft_consensus.cc:515] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:14.555744 18048 leader_election.cc:290] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:14.556232 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:14.556630 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ef84f551b343d3ad57cf6aa39cf47f" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:14.556828 18196 raft_consensus.cc:3060] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.557348 18124 raft_consensus.cc:3055] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:14.557698 18124 raft_consensus.cc:740] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:14.559381 18124 consensus_queue.cc:260] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:14.563017 18124 raft_consensus.cc:3060] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.563772 18196 raft_consensus.cc:2468] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:14.565809 18015 leader_election.cc:304] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:14.566713 19032 raft_consensus.cc:2804] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:14.567162 19032 raft_consensus.cc:697] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:18:14.568005 19032 consensus_queue.cc:237] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:14.569559 18124 raft_consensus.cc:2468] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:14.577016 17872 catalog_manager.cc:5697] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:14.590456 19025 raft_consensus.cc:993] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.590929 19031 raft_consensus.cc:1081] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:14.592546 18927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:14.593137 18927 raft_consensus.cc:493] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.593502 18927 raft_consensus.cc:3060] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.599766 18927 raft_consensus.cc:515] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.601876 18927 leader_election.cc:290] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:14.602027 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:14.602737 18124 raft_consensus.cc:3055] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:14.602761 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2861a57f9034f7fa50b7ff5bd98789b" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:14.603091 18124 raft_consensus.cc:740] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:14.603538 18196 raft_consensus.cc:3060] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.604712 18124 consensus_queue.cc:260] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.605937 18124 raft_consensus.cc:3060] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.608825 18196 raft_consensus.cc:2468] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.609894 18873 leader_election.cc:304] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:14.610577 19038 raft_consensus.cc:2804] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:14.611089 19038 raft_consensus.cc:697] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:14.611189 18124 raft_consensus.cc:2468] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:14.612085 19038 consensus_queue.cc:237] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:14.624871 17872 catalog_manager.cc:5697] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.661578 18124 raft_consensus.cc:1275] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:14.662953 19038 consensus_queue.cc:1048] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:14.671752 18121 raft_consensus.cc:1275] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.673288 19038 consensus_queue.cc:1048] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:14.683945 18196 raft_consensus.cc:1275] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:14.685405 18048 raft_consensus.cc:1275] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.685493 19038 consensus_queue.cc:1048] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:14.687963 19057 consensus_queue.cc:1048] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:14.920091 18048 raft_consensus.cc:1275] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.921407 19039 consensus_queue.cc:1048] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:14.930681 18123 raft_consensus.cc:1275] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.938028 19057 consensus_queue.cc:1048] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:14.955545 18123 raft_consensus.cc:1275] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.957604 19032 consensus_queue.cc:1048] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:14.966099 18196 raft_consensus.cc:1275] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.968205 19067 consensus_queue.cc:1048] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:15.000149 18123 raft_consensus.cc:1275] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.001335 19057 consensus_queue.cc:1048] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.010891 18196 raft_consensus.cc:1275] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.012769 19057 consensus_queue.cc:1048] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.115193 18196 raft_consensus.cc:1275] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.116750 19057 consensus_queue.cc:1048] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.125443 18123 raft_consensus.cc:1275] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.127511 19039 consensus_queue.cc:1048] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.138238 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:15.195299 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:15.229271 18123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:15.229734 18123 raft_consensus.cc:606] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:15.232992 18124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:15.233523 18124 raft_consensus.cc:606] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS fa03f4668c724e22a467d4e621f95272
I20260709 13:18:15.235567 18124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:15.235977 18124 raft_consensus.cc:606] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:15.238305 18124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fee47e66a974e48926c551ee307b756"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:15.238786 18124 raft_consensus.cc:606] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:15.241381 18200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:15.241816 18200 raft_consensus.cc:606] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Received request to transfer leadership to TS fa03f4668c724e22a467d4e621f95272
I20260709 13:18:15.243955 19049 raft_consensus.cc:993] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:15.244019 18124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:15.244405 19026 raft_consensus.cc:1081] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Signalling peer fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:15.244607 18124 raft_consensus.cc:606] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:15.245594 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:15.245694 18048 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84"
dest_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:44122
I20260709 13:18:15.246230 18048 raft_consensus.cc:493] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.246483 18048 raft_consensus.cc:3060] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.246737 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:15.252803 18048 raft_consensus.cc:515] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:15.254576 18048 leader_election.cc:290] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:15.254897 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:15.255326 18200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da457bda0979490fa4e5277fba14ee84" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:15.255450 18124 raft_consensus.cc:3060] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.255851 18200 raft_consensus.cc:3055] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:15.256141 18200 raft_consensus.cc:740] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:15.256935 18200 consensus_queue.cc:260] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:15.258177 18200 raft_consensus.cc:3060] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.261873 18124 raft_consensus.cc:2468] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:15.262955 18018 leader_election.cc:304] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:15.263770 19024 raft_consensus.cc:2804] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:15.263948 18200 raft_consensus.cc:2468] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:15.264130 19024 raft_consensus.cc:697] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:18:15.265048 19024 consensus_queue.cc:237] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:15.276494 17872 catalog_manager.cc:5697] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.291193 19031 raft_consensus.cc:993] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.291700 19031 raft_consensus.cc:1081] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.293085 18925 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:15.293551 18925 raft_consensus.cc:493] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.293875 18925 raft_consensus.cc:3060] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.300958 18925 raft_consensus.cc:515] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.302640 18925 leader_election.cc:290] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:15.303225 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:15.303608 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6165069b9c0d4a2bb1637ab3ade3fe09" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:15.303789 18048 raft_consensus.cc:3060] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.304111 18124 raft_consensus.cc:3055] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:15.304359 18124 raft_consensus.cc:740] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:15.304991 18124 consensus_queue.cc:260] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.305960 18124 raft_consensus.cc:3060] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.309144 18048 raft_consensus.cc:2468] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.312192 18124 raft_consensus.cc:2468] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.313213 18877 leader_election.cc:304] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:18:15.313954 19056 raft_consensus.cc:2804] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:15.314378 19056 raft_consensus.cc:697] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:15.315078 19056 consensus_queue.cc:237] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.324862 17872 catalog_manager.cc:5697] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.421794 19025 raft_consensus.cc:993] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.422204 19031 raft_consensus.cc:1081] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.423636 18925 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fee47e66a974e48926c551ee307b756"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:15.424034 18925 raft_consensus.cc:493] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.424307 18925 raft_consensus.cc:3060] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.430127 18925 raft_consensus.cc:515] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.432986 18925 leader_election.cc:290] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:15.433164 18124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:15.433755 18124 raft_consensus.cc:3055] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:15.433959 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fee47e66a974e48926c551ee307b756" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:15.434523 18047 raft_consensus.cc:3060] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.433966 18124 raft_consensus.cc:740] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:15.439339 18047 raft_consensus.cc:2468] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.439158 18124 consensus_queue.cc:260] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.440233 18124 raft_consensus.cc:3060] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.440551 18874 leader_election.cc:304] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:15.441398 19056 raft_consensus.cc:2804] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:15.441879 19056 raft_consensus.cc:697] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:15.442988 19056 consensus_queue.cc:237] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.449927 18124 raft_consensus.cc:2468] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.467360 17872 catalog_manager.cc:5697] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.487172 19025 raft_consensus.cc:993] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.487687 19025 raft_consensus.cc:1081] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.489022 18925 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:15.489485 18925 raft_consensus.cc:493] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.489755 18925 raft_consensus.cc:3060] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.495896 18925 raft_consensus.cc:515] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:15.497341 18925 leader_election.cc:290] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:15.497988 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:15.498160 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d07ebb0703849dbab22922f7f878958" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:15.498587 18123 raft_consensus.cc:3055] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:15.498713 18196 raft_consensus.cc:3060] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.498864 18123 raft_consensus.cc:740] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:15.499595 18123 consensus_queue.cc:260] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:15.500489 18123 raft_consensus.cc:3060] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.503690 18196 raft_consensus.cc:2468] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.504909 18873 leader_election.cc:304] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:15.505574 19056 raft_consensus.cc:2804] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:15.505908 18123 raft_consensus.cc:2468] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.505980 19056 raft_consensus.cc:697] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:15.507157 19056 consensus_queue.cc:237] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:15.515293 17872 catalog_manager.cc:5697] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.596740 19031 raft_consensus.cc:993] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230: : Instructing follower fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:15.597245 19025 raft_consensus.cc:1081] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:15.598839 18047 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91"
dest_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:44110
I20260709 13:18:15.599361 18047 raft_consensus.cc:493] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.599750 18047 raft_consensus.cc:3060] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.604630 18047 raft_consensus.cc:515] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:15.605945 18047 leader_election.cc:290] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:15.606773 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:15.606839 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "729ea2fca9fb40e5a0c2fd994a5adb91" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:15.607324 18123 raft_consensus.cc:3055] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:15.607461 18196 raft_consensus.cc:3060] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.607704 18123 raft_consensus.cc:740] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:15.608414 18123 consensus_queue.cc:260] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:15.609285 18123 raft_consensus.cc:3060] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.612102 18196 raft_consensus.cc:2468] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:15.613140 18015 leader_election.cc:304] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:15.613926 19067 raft_consensus.cc:2804] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:15.614131 18123 raft_consensus.cc:2468] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:15.614428 19067 raft_consensus.cc:697] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:18:15.615363 19067 consensus_queue.cc:237] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:15.625468 17872 catalog_manager.cc:5697] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.663627 19031 raft_consensus.cc:993] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.664535 19025 raft_consensus.cc:1081] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:15.665903 18926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:15.666451 18926 raft_consensus.cc:493] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:15.666785 18926 raft_consensus.cc:3060] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.671676 18926 raft_consensus.cc:515] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.673074 18926 leader_election.cc:290] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:15.673957 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:15.674113 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb98cd681e0496393f8af9091c9184b" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:15.674518 18047 raft_consensus.cc:3060] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.674675 18123 raft_consensus.cc:3055] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:15.675025 18123 raft_consensus.cc:740] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:15.675747 18123 consensus_queue.cc:260] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.676712 18123 raft_consensus.cc:3060] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.680054 18047 raft_consensus.cc:2468] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.683286 18874 leader_election.cc:304] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:15.684266 18123 raft_consensus.cc:2468] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:15.684487 19038 raft_consensus.cc:2804] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:15.685043 19038 raft_consensus.cc:697] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:15.685926 19038 consensus_queue.cc:237] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:15.695390 17872 catalog_manager.cc:5697] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:15.727123 18123 raft_consensus.cc:1275] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.728288 19038 consensus_queue.cc:1048] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.731455 18124 raft_consensus.cc:1275] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.733129 19024 consensus_queue.cc:1048] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.748973 18196 raft_consensus.cc:1275] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.750499 19032 consensus_queue.cc:1048] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.756038 18047 raft_consensus.cc:1275] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.757865 19056 consensus_queue.cc:1048] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.948012 18124 raft_consensus.cc:1275] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.949237 19038 consensus_queue.cc:1048] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:15.959692 18047 raft_consensus.cc:1275] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.960834 19039 consensus_queue.cc:1048] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:16.057374 18123 raft_consensus.cc:1275] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:16.058446 19039 consensus_queue.cc:1048] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:16.066830 18200 raft_consensus.cc:1275] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:16.068406 19056 consensus_queue.cc:1048] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:16.091248 18123 raft_consensus.cc:1275] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:16.093374 19056 consensus_queue.cc:1048] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:16.102897 18047 raft_consensus.cc:1275] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:16.107131 19039 consensus_queue.cc:1048] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:16.115068 18200 raft_consensus.cc:1275] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:16.131510 19098 consensus_queue.cc:1048] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:16.145460 18124 raft_consensus.cc:1275] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:16.148098 19098 consensus_queue.cc:1048] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:16.247671 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:16.289050 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:16.325417 18124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:16.325932 18124 raft_consensus.cc:606] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS fa03f4668c724e22a467d4e621f95272
I20260709 13:18:16.329134 18196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "347efb4b336e41469ded6211da2d4498"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:16.331029 18196 raft_consensus.cc:606] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:16.333969 18196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:16.334484 18196 raft_consensus.cc:606] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:16.336895 18196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:16.337338 18196 raft_consensus.cc:606] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:16.339488 18196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:16.339882 18196 raft_consensus.cc:606] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:16.340802 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:16.341955 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:16.401340 19049 raft_consensus.cc:993] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:16.401858 19079 raft_consensus.cc:1081] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:16.403363 18925 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:18:16.403828 18925 raft_consensus.cc:493] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:16.404084 18925 raft_consensus.cc:3060] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.408490 18925 raft_consensus.cc:515] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:16.409821 18925 leader_election.cc:290] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:16.410511 18123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:16.411058 18123 raft_consensus.cc:3060] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.411087 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848c8b98e4dd44e398eb0b2689ea69cd" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:16.411975 18196 raft_consensus.cc:3055] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:16.412326 18196 raft_consensus.cc:740] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:16.413218 18196 consensus_queue.cc:260] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:16.414433 18196 raft_consensus.cc:3060] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.417043 18123 raft_consensus.cc:2468] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:16.418314 18877 leader_election.cc:304] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:18:16.418980 19056 raft_consensus.cc:2804] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:16.419461 19056 raft_consensus.cc:697] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:16.420274 19056 consensus_queue.cc:237] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } }
I20260709 13:18:16.421857 18196 raft_consensus.cc:2468] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:16.429010 17871 catalog_manager.cc:5697] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:16.590346 19049 raft_consensus.cc:993] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:16.590831 19079 raft_consensus.cc:1081] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:16.592248 18926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "347efb4b336e41469ded6211da2d4498"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:18:16.592675 18926 raft_consensus.cc:493] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:16.593014 18926 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:16.598191 18926 raft_consensus.cc:515] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:16.599721 18926 leader_election.cc:290] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:18:16.600358 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:16.600733 18048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347efb4b336e41469ded6211da2d4498" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:16.601265 18048 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:16.601342 18196 raft_consensus.cc:3055] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:16.601732 18196 raft_consensus.cc:740] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:16.602427 18196 consensus_queue.cc:260] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:16.603313 18196 raft_consensus.cc:3060] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:16.621853 18048 raft_consensus.cc:2468] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 3.
I20260709 13:18:16.622560 18196 raft_consensus.cc:2468] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 3.
I20260709 13:18:16.623251 18874 leader_election.cc:304] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:16.624322 19038 raft_consensus.cc:2804] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:16.624773 19038 raft_consensus.cc:697] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:16.625573 19038 consensus_queue.cc:237] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:16.634776 19079 raft_consensus.cc:993] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:16.635386 19106 raft_consensus.cc:1081] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:16.637092 18925 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:18:16.637657 18925 raft_consensus.cc:493] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:16.637930 18925 raft_consensus.cc:3060] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.648875 18925 raft_consensus.cc:515] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:16.686621 18925 leader_election.cc:290] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:18:16.693552 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:16.694226 18196 raft_consensus.cc:3055] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:16.694558 18196 raft_consensus.cc:740] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:16.697469 18196 consensus_queue.cc:260] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:16.698987 18196 raft_consensus.cc:3060] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.350509 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fd2d32e1713467387bb231dc639c10d" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:17.351176 18047 raft_consensus.cc:3060] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.354261 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:17.362397 18047 raft_consensus.cc:2468] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:17.388614 18874 leader_election.cc:304] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:17.389953 19056 raft_consensus.cc:2804] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:17.401209 19056 raft_consensus.cc:697] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:17.402398 19056 consensus_queue.cc:237] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:17.452005 18050 raft_consensus.cc:1275] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:17.464938 19118 consensus_queue.cc:1048] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:17.467744 18122 raft_consensus.cc:1275] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:17.487191 19118 consensus_queue.cc:1048] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:18:17.509043 18196 consensus_meta.cc:287] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: Time spent flushing consensus metadata: real 0.809s	user 0.014s	sys 0.003s
I20260709 13:18:17.509410 19119 raft_consensus.cc:993] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230: : Instructing follower fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:17.509995 19025 raft_consensus.cc:1081] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:17.514292 18049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506"
dest_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:44110
I20260709 13:18:17.514875 18049 raft_consensus.cc:493] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.515262 18049 raft_consensus.cc:3060] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.527446 18198 raft_consensus.cc:1275] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:17.533097 18198 raft_consensus.cc:1275] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:17.533058 18164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.66:40273, user_credentials={real_user=slave}} blocked reactor thread for 63105.3us
I20260709 13:18:17.542215 19115 raft_consensus.cc:993] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:17.543148 19079 raft_consensus.cc:1081] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:17.543740 19056 consensus_queue.cc:1048] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:17.544875 17875 catalog_manager.cc:5697] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 2 to 3, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 3 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } }
I20260709 13:18:17.546893 18196 raft_consensus.cc:2468] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:17.554523 18921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:18:17.555166 18921 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.555541 18921 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:17.564951 18049 raft_consensus.cc:515] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.565649 19039 consensus_queue.cc:1048] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:17.581702 18049 leader_election.cc:290] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:17.583626 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:17.586416 18122 raft_consensus.cc:3055] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:17.586282 18921 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:17.586803 18122 raft_consensus.cc:740] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:17.584830 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15a02bb6e41942bba7f6cc5358ef1506" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:17.608494 18196 raft_consensus.cc:3060] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.617970 18196 raft_consensus.cc:2468] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:17.600396 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:17.619470 18015 leader_election.cc:304] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:17.620504 19109 raft_consensus.cc:2804] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:17.620929 19109 raft_consensus.cc:697] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:18:17.621824 19109 consensus_queue.cc:237] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.622325 18122 consensus_queue.cc:260] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.624245 18122 raft_consensus.cc:3060] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.627861 18049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:17.620838 18921 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:18:17.628468 18049 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:17.636472 18873 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.68:33171, user_credentials={real_user=slave}} blocked reactor thread for 89681.1us
I20260709 13:18:17.638183 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:17.638824 18196 raft_consensus.cc:3055] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:17.639153 18196 raft_consensus.cc:740] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:17.640054 18196 consensus_queue.cc:260] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:17.642592 18196 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:17.642982 18122 raft_consensus.cc:2468] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:17.647099 18049 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 3.
I20260709 13:18:17.648905 18874 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:17.649816 19039 raft_consensus.cc:2804] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:17.650261 19039 raft_consensus.cc:697] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:17.651137 19039 consensus_queue.cc:237] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:17.658440 18196 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 3.
I20260709 13:18:17.672345 17871 catalog_manager.cc:5697] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.681667 17875 catalog_manager.cc:5697] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.703272 18200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:17.703862 18200 raft_consensus.cc:606] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:17.709232 18200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "1787628aa7f6489497bb871588797230"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:17.710759 17871 catalog_manager.cc:5697] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 2 to 3, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 3 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.712944 18200 raft_consensus.cc:606] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Received request to transfer leadership to TS 1787628aa7f6489497bb871588797230
I20260709 13:18:17.716778 18198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
mode: GRACEFUL
new_leader_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:49262
I20260709 13:18:17.717237 18198 raft_consensus.cc:606] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Received request to transfer leadership to TS fa03f4668c724e22a467d4e621f95272
I20260709 13:18:17.719830 18122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8"
dest_uuid: "1787628aa7f6489497bb871588797230"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:52870
I20260709 13:18:17.720391 18122 raft_consensus.cc:606] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
W20260709 13:18:17.720651 17906 debug-util.cc:398] Leaking SignalData structure 0x7b080014c3e0 after lost signal to thread 10260
I20260709 13:18:17.721717 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:18:17.721817 17906 debug-util.cc:398] Leaking SignalData structure 0x7b080016c600 after lost signal to thread 17909
I20260709 13:18:17.724133 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:18:17.724372 17906 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2220 after lost signal to thread 18076
W20260709 13:18:17.725735 17906 debug-util.cc:398] Leaking SignalData structure 0x7b0800251660 after lost signal to thread 18150
W20260709 13:18:17.726975 17906 debug-util.cc:398] Leaking SignalData structure 0x7b0800375620 after lost signal to thread 18224
W20260709 13:18:17.728271 17906 debug-util.cc:398] Leaking SignalData structure 0x7b0800241bc0 after lost signal to thread 18954
I20260709 13:18:17.730387 19114 raft_consensus.cc:993] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:17.730981 19122 raft_consensus.cc:1081] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:17.732790 18922 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40722
I20260709 13:18:17.733402 18922 raft_consensus.cc:493] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.733754 18922 raft_consensus.cc:3060] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.741255 18922 raft_consensus.cc:515] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:17.742651 19122 raft_consensus.cc:993] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower 1787628aa7f6489497bb871588797230 to start an election
I20260709 13:18:17.743762 19122 raft_consensus.cc:1081] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Signalling peer 1787628aa7f6489497bb871588797230 to start an election
I20260709 13:18:17.744577 19122 raft_consensus.cc:993] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568: : Instructing follower fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:17.745013 19079 raft_consensus.cc:1081] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Signalling peer fa03f4668c724e22a467d4e621f95272 to start an election
I20260709 13:18:17.745517 18122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f"
dest_uuid: "1787628aa7f6489497bb871588797230"
 from {username='slave'} at 127.0.0.1:39902
I20260709 13:18:17.746101 18122 raft_consensus.cc:493] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.746558 18122 raft_consensus.cc:3060] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.746798 18049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6"
dest_uuid: "fa03f4668c724e22a467d4e621f95272"
 from {username='slave'} at 127.0.0.1:44122
I20260709 13:18:17.747383 18922 leader_election.cc:290] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:18:17.747381 18049 raft_consensus.cc:493] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.748103 18049 raft_consensus.cc:3060] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.748227 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:17.748838 18047 raft_consensus.cc:3060] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.764580 18122 raft_consensus.cc:515] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.765431 18047 raft_consensus.cc:2468] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:17.767099 18874 leader_election.cc:304] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:17.765348 18049 raft_consensus.cc:515] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.768093 19128 raft_consensus.cc:2804] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:17.768157 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:17.768787 19128 raft_consensus.cc:697] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:17.769279 18196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6200167e8cfc45a59b2b043a2601401f" candidate_uuid: "1787628aa7f6489497bb871588797230" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:17.768818 18050 raft_consensus.cc:3060] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.769855 18196 raft_consensus.cc:3055] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:17.770277 18196 raft_consensus.cc:740] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:17.771044 18196 consensus_queue.cc:260] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.769795 19128 consensus_queue.cc:237] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:17.772154 18196 raft_consensus.cc:3060] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.777648 18050 raft_consensus.cc:2468] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:18:17.779167 18090 leader_election.cc:304] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:17.769765 18122 leader_election.cc:290] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:17.783298 18049 leader_election.cc:290] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Requested vote from peers 1787628aa7f6489497bb871588797230 (127.10.4.67:43399), fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68:33171)
I20260709 13:18:17.783556 19025 raft_consensus.cc:2804] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:17.784055 19025 raft_consensus.cc:697] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Becoming Leader. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:17.784447 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:17.785157 19111 raft_consensus.cc:993] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:17.785990 19111 raft_consensus.cc:1081] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:17.787035 18122 raft_consensus.cc:3060] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.784759 19025 consensus_queue.cc:237] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.788278 18195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65f480c3ad843578f87c43aee4ecbc6" candidate_uuid: "fa03f4668c724e22a467d4e621f95272" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:17.788848 18195 raft_consensus.cc:3055] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:17.789127 18195 raft_consensus.cc:740] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:17.789237 18196 raft_consensus.cc:2468] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1787628aa7f6489497bb871588797230 in term 2.
I20260709 13:18:17.790035 18195 consensus_queue.cc:260] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.792600 18924 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40730
I20260709 13:18:17.793180 18924 raft_consensus.cc:493] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:17.793488 18122 raft_consensus.cc:2468] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:17.793511 18924 raft_consensus.cc:3060] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.791366 18195 raft_consensus.cc:3060] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.806857 17872 catalog_manager.cc:5697] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 reported cstate change: term changed from 1 to 2, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 1787628aa7f6489497bb871588797230 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "1787628aa7f6489497bb871588797230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.809527 18924 raft_consensus.cc:515] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:17.754737 18198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa2530e8b624041bc011fa928d7078f" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
I20260709 13:18:17.811640 18198 raft_consensus.cc:3055] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:17.811931 18198 raft_consensus.cc:740] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:17.812659 18198 consensus_queue.cc:260] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:17.813596 18924 leader_election.cc:290] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:17.814551 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:17.812639 18047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb3ec66980be4a6ea8be1b3fa67292b8" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:17.815531 18047 raft_consensus.cc:3060] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.815266 18122 raft_consensus.cc:3055] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:17.818730 18122 raft_consensus.cc:740] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1787628aa7f6489497bb871588797230, State: Running, Role: LEADER
I20260709 13:18:17.818894 18018 leader_election.cc:304] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:17.819744 18122 consensus_queue.cc:260] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:17.814884 18198 raft_consensus.cc:3060] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.823556 19067 raft_consensus.cc:2804] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:17.824070 19067 raft_consensus.cc:697] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Becoming Leader. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:18:17.820672 17871 catalog_manager.cc:5697] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.824877 19067 consensus_queue.cc:237] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:17.824084 18122 raft_consensus.cc:3060] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.823174 18047 raft_consensus.cc:2468] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:17.848954 18874 leader_election.cc:304] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242d2387755e4a37ade4260e80f4eb6a, fa03f4668c724e22a467d4e621f95272; no voters: 
I20260709 13:18:17.844576 18049 raft_consensus.cc:1275] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:17.850487 18122 raft_consensus.cc:2468] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:17.845635 18195 raft_consensus.cc:2468] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa03f4668c724e22a467d4e621f95272 in term 2.
I20260709 13:18:17.855767 17871 catalog_manager.cc:5697] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 reported cstate change: term changed from 1 to 2, leader changed from fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68) to fa03f4668c724e22a467d4e621f95272 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "fa03f4668c724e22a467d4e621f95272" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.831931 18198 raft_consensus.cc:2468] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:17.856665 19110 raft_consensus.cc:2804] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:17.865355 19110 raft_consensus.cc:697] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:17.866262 19110 consensus_queue.cc:237] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:17.857621 19128 consensus_queue.cc:1048] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:17.886356 17872 catalog_manager.cc:5697] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from 1787628aa7f6489497bb871588797230 (127.10.4.67) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.928869 18200 raft_consensus.cc:1275] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:17.930256 19118 consensus_queue.cc:1048] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.056131 18198 raft_consensus.cc:1275] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:18.057358 19110 consensus_queue.cc:1048] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.067333 18050 raft_consensus.cc:1275] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:18.069669 19038 consensus_queue.cc:1048] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.085671 18198 raft_consensus.cc:1275] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.087774 19108 consensus_queue.cc:1048] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.108795 18122 raft_consensus.cc:1275] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.110329 19109 consensus_queue.cc:1048] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.200724 18050 raft_consensus.cc:1275] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 1787628aa7f6489497bb871588797230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.202492 19025 consensus_queue.cc:1048] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.225802 18200 raft_consensus.cc:1275] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 1787628aa7f6489497bb871588797230: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.227888 19031 consensus_queue.cc:1048] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.261371 18050 raft_consensus.cc:1275] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.263556 19038 consensus_queue.cc:1048] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.287930 18198 raft_consensus.cc:1275] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.291502 19038 consensus_queue.cc:1048] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.312726 18122 raft_consensus.cc:1275] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.314962 19128 consensus_queue.cc:1048] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.336902 18050 raft_consensus.cc:1275] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.338692 19110 consensus_queue.cc:1048] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.380620 18200 raft_consensus.cc:1275] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.381789 19098 consensus_queue.cc:1048] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:18.394601 18122 raft_consensus.cc:1275] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fa03f4668c724e22a467d4e621f95272: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.399135 19109 consensus_queue.cc:1048] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.725147 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:18.776651 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:18.826373 18050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce"
dest_uuid: "fa03f4668c724e22a467d4e621f95272"
mode: GRACEFUL
new_leader_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:45476
I20260709 13:18:18.826905 18050 raft_consensus.cc:606] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Received request to transfer leadership to TS 242d2387755e4a37ade4260e80f4eb6a
I20260709 13:18:18.829425 18050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c"
dest_uuid: "fa03f4668c724e22a467d4e621f95272"
mode: GRACEFUL
new_leader_uuid: "fc8a9ec31a5745d69dae4262b271e568"
 from {username='slave'} at 127.0.0.1:45476
I20260709 13:18:18.829878 18050 raft_consensus.cc:606] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Received request to transfer leadership to TS fc8a9ec31a5745d69dae4262b271e568
I20260709 13:18:18.830859 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:18.832176 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:19.062915 19067 raft_consensus.cc:993] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: : Instructing follower fc8a9ec31a5745d69dae4262b271e568 to start an election
I20260709 13:18:19.063201 19109 raft_consensus.cc:993] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272: : Instructing follower 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:19.063422 19108 raft_consensus.cc:1081] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Signalling peer fc8a9ec31a5745d69dae4262b271e568 to start an election
I20260709 13:18:19.063815 19067 raft_consensus.cc:1081] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Signalling peer 242d2387755e4a37ade4260e80f4eb6a to start an election
I20260709 13:18:19.064785 18198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:18:19.065265 18198 raft_consensus.cc:493] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:19.065217 18921 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
 from {username='slave'} at 127.0.0.1:40712
I20260709 13:18:19.065848 18198 raft_consensus.cc:3060] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:19.066051 18921 raft_consensus.cc:493] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:19.066483 18921 raft_consensus.cc:3060] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:19.070600 18198 raft_consensus.cc:515] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:19.071271 18921 raft_consensus.cc:515] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:19.073609 18122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:19.073668 18921 leader_election.cc:290] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:19.074620 18052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3445b301b2410eba0473bd236b619c" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:19.075004 18122 raft_consensus.cc:3060] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:19.075327 18052 raft_consensus.cc:3055] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:19.075666 18052 raft_consensus.cc:740] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:18:19.073944 18126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1787628aa7f6489497bb871588797230"
I20260709 13:18:19.076242 18126 raft_consensus.cc:3060] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:19.076465 18052 consensus_queue.cc:260] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:19.077512 18052 raft_consensus.cc:3060] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:19.074743 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017bc4b9de4c4462845e7b860bfa6dce" candidate_uuid: "242d2387755e4a37ade4260e80f4eb6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:19.079628 18050 raft_consensus.cc:3055] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:19.079938 18050 raft_consensus.cc:740] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa03f4668c724e22a467d4e621f95272, State: Running, Role: LEADER
I20260709 13:18:19.080461 18122 raft_consensus.cc:2468] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:18:19.080616 18050 consensus_queue.cc:260] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:19.081869 18166 leader_election.cc:304] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:19.081869 18050 raft_consensus.cc:3060] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:19.082814 19155 raft_consensus.cc:2804] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:19.082836 18126 raft_consensus.cc:2468] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:19.083345 19155 raft_consensus.cc:697] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:19.084183 18877 leader_election.cc:304] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1787628aa7f6489497bb871588797230, 242d2387755e4a37ade4260e80f4eb6a; no voters: 
I20260709 13:18:19.084241 19155 consensus_queue.cc:237] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } }
I20260709 13:18:19.085011 19039 raft_consensus.cc:2804] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:19.085505 19039 raft_consensus.cc:697] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Becoming Leader. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:19.086293 19039 consensus_queue.cc:237] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } }
I20260709 13:18:19.086839 18198 leader_election.cc:290] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 2 election: Requested vote from peers fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273), 1787628aa7f6489497bb871588797230 (127.10.4.67:43399)
I20260709 13:18:19.088719 18052 raft_consensus.cc:2468] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 2.
I20260709 13:18:19.093488 18050 raft_consensus.cc:2468] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242d2387755e4a37ade4260e80f4eb6a in term 2.
I20260709 13:18:19.098634 17871 catalog_manager.cc:5697] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a reported cstate change: term changed from 1 to 2, leader changed from fa03f4668c724e22a467d4e621f95272 (127.10.4.66) to 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65). New cstate: current_term: 2 leader_uuid: "242d2387755e4a37ade4260e80f4eb6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:19.100754 17875 catalog_manager.cc:5697] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 1 to 2, leader changed from fa03f4668c724e22a467d4e621f95272 (127.10.4.66) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 2 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:19.490674 18122 raft_consensus.cc:1275] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:19.491672 19122 consensus_queue.cc:1048] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:19.500705 18050 raft_consensus.cc:1275] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:19.501976 19122 consensus_queue.cc:1048] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:19.629635 18050 raft_consensus.cc:1275] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:19.630877 19110 consensus_queue.cc:1048] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:19.645440 18122 raft_consensus.cc:1275] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Refusing update from remote peer 242d2387755e4a37ade4260e80f4eb6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:19.647017 19110 consensus_queue.cc:1048] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1787628aa7f6489497bb871588797230" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:19.833053 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:19.889504 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:19.939169 18921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2"
dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
mode: GRACEFUL
new_leader_uuid: "fc8a9ec31a5745d69dae4262b271e568"
 from {username='slave'} at 127.0.0.1:33934
I20260709 13:18:19.939756 18921 raft_consensus.cc:606] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Received request to transfer leadership to TS fc8a9ec31a5745d69dae4262b271e568
I20260709 13:18:19.941035 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:18:19.942637 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:19.966609 19173 raft_consensus.cc:993] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a: : Instructing follower fc8a9ec31a5745d69dae4262b271e568 to start an election
I20260709 13:18:19.967128 19110 raft_consensus.cc:1081] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Signalling peer fc8a9ec31a5745d69dae4262b271e568 to start an election
I20260709 13:18:19.968708 18195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2"
dest_uuid: "fc8a9ec31a5745d69dae4262b271e568"
 from {username='slave'} at 127.0.0.1:49266
I20260709 13:18:19.969148 18195 raft_consensus.cc:493] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:19.969442 18195 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.975215 18195 raft_consensus.cc:515] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:19.977952 18195 leader_election.cc:290] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 election: Requested vote from peers 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893), fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273)
I20260709 13:18:19.978086 18050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fa03f4668c724e22a467d4e621f95272"
I20260709 13:18:19.978731 18050 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.980891 18921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4c004aeb9af431087cd2e10becd16c2" candidate_uuid: "fc8a9ec31a5745d69dae4262b271e568" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "242d2387755e4a37ade4260e80f4eb6a"
I20260709 13:18:19.981424 18921 raft_consensus.cc:3055] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:19.981664 18921 raft_consensus.cc:740] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 242d2387755e4a37ade4260e80f4eb6a, State: Running, Role: LEADER
I20260709 13:18:19.982297 18921 consensus_queue.cc:260] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:19.983217 18921 raft_consensus.cc:3060] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.984089 18050 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 4.
I20260709 13:18:19.985184 18164 leader_election.cc:304] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa03f4668c724e22a467d4e621f95272, fc8a9ec31a5745d69dae4262b271e568; no voters: 
I20260709 13:18:19.985899 19122 raft_consensus.cc:2804] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:19.986389 19122 raft_consensus.cc:697] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 4 LEADER]: Becoming Leader. State: Replica: fc8a9ec31a5745d69dae4262b271e568, State: Running, Role: LEADER
I20260709 13:18:19.987193 19122 consensus_queue.cc:237] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } }
I20260709 13:18:19.990496 18921 raft_consensus.cc:2468] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc8a9ec31a5745d69dae4262b271e568 in term 4.
I20260709 13:18:19.999608 17871 catalog_manager.cc:5697] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 reported cstate change: term changed from 3 to 4, leader changed from 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65) to fc8a9ec31a5745d69dae4262b271e568 (127.10.4.68). New cstate: current_term: 4 leader_uuid: "fc8a9ec31a5745d69dae4262b271e568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc8a9ec31a5745d69dae4262b271e568" member_type: VOTER last_known_addr { host: "127.10.4.68" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:20.391727 18052 raft_consensus.cc:1275] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 4 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:20.395540 19184 consensus_queue.cc:1048] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa03f4668c724e22a467d4e621f95272" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:20.408197 18921 raft_consensus.cc:1275] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 4 FOLLOWER]: Refusing update from remote peer fc8a9ec31a5745d69dae4262b271e568: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:20.412712 19184 consensus_queue.cc:1048] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242d2387755e4a37ade4260e80f4eb6a" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:20.943579 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:20.978706 10257 tablet_server.cc:179] TabletServer@127.10.4.65:32893 shutting down...
W20260709 13:18:20.994442 18091 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 295 similar messages]
W20260709 13:18:21.000104 18091 consensus_peers.cc:597] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.012656 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:21.013500 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.014518 10257 raft_consensus.cc:2243] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.014911 10257 raft_consensus.cc:2272] T dd76d33e673a4966a9c91b362fafb0b1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.017136 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.017767 10257 raft_consensus.cc:2243] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.018504 10257 raft_consensus.cc:2272] T 3fd2d32e1713467387bb231dc639c10d P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.020514 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.021062 10257 raft_consensus.cc:2243] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.021469 10257 raft_consensus.cc:2272] T 72bffd5492454e96ae4e10907b97381c P 242d2387755e4a37ade4260e80f4eb6a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.023788 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.024649 10257 raft_consensus.cc:2243] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.025660 10257 raft_consensus.cc:2272] T 017bc4b9de4c4462845e7b860bfa6dce P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.027041 18017 consensus_peers.cc:597] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.029425 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.030586 10257 raft_consensus.cc:2243] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.031672 10257 raft_consensus.cc:2272] T 8fee47e66a974e48926c551ee307b756 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.034461 10257 tablet_replica.cc:333] stopping tablet replica
W20260709 13:18:21.034214 18165 consensus_peers.cc:597] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.035358 10257 raft_consensus.cc:2243] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.035799 10257 raft_consensus.cc:2272] T 6247cfa8622e49e29ad0bfba76525834 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.037911 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.038476 10257 raft_consensus.cc:2243] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.038859 10257 raft_consensus.cc:2272] T f0771f9326ce4105929b0fca4a301e1e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.041061 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.041627 10257 raft_consensus.cc:2243] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.042021 10257 raft_consensus.cc:2272] T 110dca2dfd484809b9e058a7ab22b49a P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.044201 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.044996 10257 raft_consensus.cc:2243] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.045499 10257 raft_consensus.cc:2272] T 211547823b99438c9cc10432cc919a5f P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.048261 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.049034 10257 raft_consensus.cc:2243] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.049566 10257 raft_consensus.cc:2272] T 00cc4ceb4dee449a8998229f3bd78288 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.052078 18091 consensus_peers.cc:597] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.052412 10257 tablet_replica.cc:333] stopping tablet replica
W20260709 13:18:21.052739 18091 consensus_peers.cc:597] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.053300 10257 raft_consensus.cc:2243] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.053696 10257 raft_consensus.cc:2272] T 843fde44d03641ce89e8d2e247c68b2d P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.055641 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.056133 10257 raft_consensus.cc:2243] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.056491 10257 raft_consensus.cc:2272] T 46d0d9f275f44ae59f4f6c2df48acd6d P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.058285 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.058765 10257 raft_consensus.cc:2243] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.059084 10257 raft_consensus.cc:2272] T 127bfde2c3624e5bb2d0f68929889394 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.060986 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.061582 10257 raft_consensus.cc:2243] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.062096 10257 raft_consensus.cc:2272] T a80b08582e1a4e099492c0650e820c45 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.064651 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.065272 10257 raft_consensus.cc:2243] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.065598 10257 raft_consensus.cc:2272] T 020d6114c75749cfab1d9c0b047da996 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.067695 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.068221 10257 raft_consensus.cc:2243] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.068884 10257 raft_consensus.cc:2272] T 8989b7b66db644f89f6c4421fd51f2cc P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.070626 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.071107 10257 raft_consensus.cc:2243] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.071601 10257 raft_consensus.cc:2272] T f6de10a3b7bb4a8da2902e5f42108ed2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.073674 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.074203 10257 raft_consensus.cc:2243] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.074611 10257 raft_consensus.cc:2272] T c143c20585e74a46807a580f6dcdafc1 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.077006 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.077704 10257 raft_consensus.cc:2243] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.078101 10257 raft_consensus.cc:2272] T d4bd29ab75004ddf9b1ce255a071ed24 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.080070 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.080534 10257 raft_consensus.cc:2243] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.080868 10257 raft_consensus.cc:2272] T 3ad41548503545059d74bd2564c17b52 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.082723 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.083191 10257 raft_consensus.cc:2243] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.083508 10257 raft_consensus.cc:2272] T a7241c7eca1b4c53a4faea0bcffee354 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.085256 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.085709 10257 raft_consensus.cc:2243] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.086352 10257 raft_consensus.cc:2272] T 9f4120b61f1d44f6a4a3c5c4575f993a P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.088171 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.088666 10257 raft_consensus.cc:2243] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.089051 10257 raft_consensus.cc:2272] T a4c004aeb9af431087cd2e10becd16c2 P 242d2387755e4a37ade4260e80f4eb6a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.091620 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.092191 10257 raft_consensus.cc:2243] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.092588 10257 raft_consensus.cc:2272] T 30604e2cd10b450fb3e8d52bee20f8da P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.094518 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.095000 10257 raft_consensus.cc:2243] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.095741 10257 raft_consensus.cc:2272] T b2861a57f9034f7fa50b7ff5bd98789b P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.097970 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.098696 10257 raft_consensus.cc:2243] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.099668 10257 raft_consensus.cc:2272] T 79c18e4eea964029ad14dc02ce2137e2 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.102273 10257 tablet_replica.cc:333] stopping tablet replica
W20260709 13:18:21.102344 18165 consensus_peers.cc:597] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.102843 10257 raft_consensus.cc:2243] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.103343 10257 raft_consensus.cc:2272] T 0044e0ba4e7548ea8cd3452eed875fa8 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.105413 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.105952 10257 raft_consensus.cc:2243] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.106312 10257 raft_consensus.cc:2272] T e1606bb8512a4c5ba5133be786298984 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.108327 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.108814 10257 raft_consensus.cc:2243] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.109515 10257 raft_consensus.cc:2272] T 347efb4b336e41469ded6211da2d4498 P 242d2387755e4a37ade4260e80f4eb6a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.111526 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.112167 10257 raft_consensus.cc:2243] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.112725 10257 raft_consensus.cc:2272] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.114893 18165 consensus_peers.cc:597] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.115515 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.116375 10257 raft_consensus.cc:2243] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.117285 10257 raft_consensus.cc:2272] T fcb98cd681e0496393f8af9091c9184b P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.119733 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.120283 10257 raft_consensus.cc:2243] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.121060 10257 raft_consensus.cc:2272] T 0aa2530e8b624041bc011fa928d7078f P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.122924 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.123452 10257 raft_consensus.cc:2243] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.123836 10257 raft_consensus.cc:2272] T bc6bc2204cfe4f09a8f0bae7240822d8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.125753 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.126361 10257 raft_consensus.cc:2243] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.127058 10257 raft_consensus.cc:2272] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.128976 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.129868 10257 raft_consensus.cc:2243] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.130805 10257 raft_consensus.cc:2272] T 7d07ebb0703849dbab22922f7f878958 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.133271 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.133957 10257 raft_consensus.cc:2243] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.134450 10257 raft_consensus.cc:2272] T f26de913e9ce4dad845f7173d82adfc1 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.135335 18091 consensus_peers.cc:597] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.137619 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.138414 10257 raft_consensus.cc:2243] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.138859 10257 raft_consensus.cc:2272] T d4cca6423ee4465cb73609e4e335e66e P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.141407 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.141976 10257 raft_consensus.cc:2243] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.142717 10257 raft_consensus.cc:2272] T 1c67ac9ecbb841cea07be431d6a78d54 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.145095 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.145654 10257 raft_consensus.cc:2243] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.146119 10257 raft_consensus.cc:2272] T 4bd55fcdd2224c1db74729d68acc0ee6 P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.148511 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.149000 10257 raft_consensus.cc:2243] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.149811 10257 raft_consensus.cc:2272] T eb3ec66980be4a6ea8be1b3fa67292b8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.151953 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.152441 10257 raft_consensus.cc:2243] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.152829 10257 raft_consensus.cc:2272] T c9de6c54e6314854955dc74a83ea79d3 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.154670 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.155303 10257 raft_consensus.cc:2243] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.156204 10257 raft_consensus.cc:2272] T 848c8b98e4dd44e398eb0b2689ea69cd P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.158536 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.159024 10257 raft_consensus.cc:2243] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.159543 10257 raft_consensus.cc:2272] T b7077b3a2710489282f1c1cd3f8c1ddb P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.162166 10257 tablet_replica.cc:333] stopping tablet replica
W20260709 13:18:21.162744 18017 consensus_peers.cc:597] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.162864 10257 raft_consensus.cc:2243] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.163585 10257 raft_consensus.cc:2272] T b8f2c37906b94d31b85748436677deef P 242d2387755e4a37ade4260e80f4eb6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.166079 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:21.166554 10257 raft_consensus.cc:2243] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.166965 10257 raft_consensus.cc:2272] T 7187dfc4efc34078a81a1f59637521c8 P 242d2387755e4a37ade4260e80f4eb6a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.209154 18017 consensus_peers.cc:597] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.214056 18165 consensus_peers.cc:597] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.235793 10257 tablet_server.cc:196] TabletServer@127.10.4.65:32893 shutdown complete.
W20260709 13:18:21.255353 18091 consensus_peers.cc:597] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.256254 18017 consensus_peers.cc:597] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.298653 18165 consensus_peers.cc:597] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.310196 18091 consensus_peers.cc:597] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.313230 18165 consensus_peers.cc:597] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.324410 18165 consensus_peers.cc:597] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.331660 18017 consensus_peers.cc:597] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.333920 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
W20260709 13:18:21.338580 18165 consensus_peers.cc:597] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.339357 18017 consensus_peers.cc:597] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.340153 18090 consensus_peers.cc:597] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.344094 18091 consensus_peers.cc:597] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.350167 18165 consensus_peers.cc:597] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.372959 18017 consensus_peers.cc:597] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.373510 18090 consensus_peers.cc:597] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.377909 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:21.378700 10257 tablet_replica.cc:333] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.379562 10257 raft_consensus.cc:2243] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.380326 10257 raft_consensus.cc:2272] T 4bd55fcdd2224c1db74729d68acc0ee6 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.382530 10257 tablet_replica.cc:333] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.383368 10257 raft_consensus.cc:2243] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.384411 10257 raft_consensus.cc:2272] T d4bd29ab75004ddf9b1ce255a071ed24 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.386092 18091 consensus_peers.cc:597] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.390391 10257 tablet_replica.cc:333] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
W20260709 13:18:21.390991 18165 consensus_peers.cc:597] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.391441 10257 raft_consensus.cc:2243] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:18:21.391597 18165 consensus_peers.cc:597] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.392098 10257 raft_consensus.cc:2272] T 020d6114c75749cfab1d9c0b047da996 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.394340 18090 consensus_peers.cc:597] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.394899 10257 tablet_replica.cc:333] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.395784 10257 raft_consensus.cc:2243] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.396718 10257 raft_consensus.cc:2272] T f1ef84f551b343d3ad57cf6aa39cf47f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.396732 18090 consensus_peers.cc:597] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.397290 18017 consensus_peers.cc:597] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.398957 10257 tablet_replica.cc:333] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.399755 10257 raft_consensus.cc:2243] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.400590 10257 raft_consensus.cc:2272] T 110dca2dfd484809b9e058a7ab22b49a P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.402736 10257 tablet_replica.cc:333] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.403318 10257 raft_consensus.cc:2243] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.404371 10257 raft_consensus.cc:2272] T b8f2c37906b94d31b85748436677deef P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.406657 10257 tablet_replica.cc:333] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.407270 10257 raft_consensus.cc:2243] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.407747 10257 raft_consensus.cc:2272] T 8fee47e66a974e48926c551ee307b756 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.409746 10257 tablet_replica.cc:333] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.410341 10257 raft_consensus.cc:2243] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.410735 10257 raft_consensus.cc:2272] T eb3ec66980be4a6ea8be1b3fa67292b8 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.412698 10257 tablet_replica.cc:333] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.413314 10257 raft_consensus.cc:2243] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.414145 10257 raft_consensus.cc:2272] T 211547823b99438c9cc10432cc919a5f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.416154 10257 tablet_replica.cc:333] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.416836 10257 raft_consensus.cc:2243] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.417315 10257 raft_consensus.cc:2272] T 691cfc20fdd444b3b53db6fdcdedcda8 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.419642 10257 tablet_replica.cc:333] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.420255 10257 raft_consensus.cc:2243] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.420778 10257 raft_consensus.cc:2272] T a4c004aeb9af431087cd2e10becd16c2 P fa03f4668c724e22a467d4e621f95272 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.423465 10257 tablet_replica.cc:333] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.424191 10257 raft_consensus.cc:2243] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.424577 10257 raft_consensus.cc:2272] T 3918cfe707c24309bd3a731cca5bcbef P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.427240 10257 tablet_replica.cc:333] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.427979 10257 raft_consensus.cc:2243] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.429059 10257 raft_consensus.cc:2272] T f6de10a3b7bb4a8da2902e5f42108ed2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.431891 10257 tablet_replica.cc:333] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.436012 10257 raft_consensus.cc:2243] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.437069 10257 raft_consensus.cc:2272] T 8989b7b66db644f89f6c4421fd51f2cc P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.439344 10257 tablet_replica.cc:333] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.439961 10257 raft_consensus.cc:2243] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.440441 10257 raft_consensus.cc:2272] T d97db5bc48fc437fac40454dc3e1d871 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.443552 10257 tablet_replica.cc:333] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.444399 10257 raft_consensus.cc:2243] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.444944 10257 raft_consensus.cc:2272] T 6200167e8cfc45a59b2b043a2601401f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.446907 18091 consensus_peers.cc:597] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.448563 10257 tablet_replica.cc:333] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.450232 10257 raft_consensus.cc:2243] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.451794 10257 raft_consensus.cc:2272] T da457bda0979490fa4e5277fba14ee84 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.452752 18090 consensus_peers.cc:597] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.454165 10257 tablet_replica.cc:333] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.454847 10257 raft_consensus.cc:2243] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.455752 10257 raft_consensus.cc:2272] T f0771f9326ce4105929b0fca4a301e1e P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.457873 10257 tablet_replica.cc:333] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.458490 10257 raft_consensus.cc:2243] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.458985 10257 raft_consensus.cc:2272] T 347efb4b336e41469ded6211da2d4498 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.462024 10257 tablet_replica.cc:333] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.462883 10257 raft_consensus.cc:2243] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.463550 10257 raft_consensus.cc:2272] T e1606bb8512a4c5ba5133be786298984 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.465991 10257 tablet_replica.cc:333] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.466691 10257 raft_consensus.cc:2243] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.467568 10257 raft_consensus.cc:2272] T 15a02bb6e41942bba7f6cc5358ef1506 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.469758 10257 tablet_replica.cc:333] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.470342 10257 raft_consensus.cc:2243] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.471161 10257 raft_consensus.cc:2272] T 127bfde2c3624e5bb2d0f68929889394 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.473348 10257 tablet_replica.cc:333] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.473915 10257 raft_consensus.cc:2243] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.474362 10257 raft_consensus.cc:2272] T 3fd2d32e1713467387bb231dc639c10d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.476608 10257 tablet_replica.cc:333] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.477244 10257 raft_consensus.cc:2243] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.477810 10257 raft_consensus.cc:2272] T a80b08582e1a4e099492c0650e820c45 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.480324 10257 tablet_replica.cc:333] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.481156 10257 raft_consensus.cc:2243] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:18:21.481520 18164 consensus_peers.cc:597] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.481886 18090 consensus_peers.cc:597] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.482178 10257 raft_consensus.cc:2272] T c65f480c3ad843578f87c43aee4ecbc6 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.482829 18091 consensus_peers.cc:597] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 -> Peer 242d2387755e4a37ade4260e80f4eb6a (127.10.4.65:32893): Couldn't send request to peer 242d2387755e4a37ade4260e80f4eb6a. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:32893: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.485045 10257 tablet_replica.cc:333] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.486083 10257 raft_consensus.cc:2243] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.487190 10257 raft_consensus.cc:2272] T 0044e0ba4e7548ea8cd3452eed875fa8 P fa03f4668c724e22a467d4e621f95272 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.489876 10257 tablet_replica.cc:333] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.490769 10257 raft_consensus.cc:2243] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.491349 10257 raft_consensus.cc:2272] T 285967e1504c483dac972f54a3d59bfc P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.493907 10257 tablet_replica.cc:333] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.494474 10257 raft_consensus.cc:2243] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.494866 10257 raft_consensus.cc:2272] T 0745baef9dbd4cd0a943445a5aa9cc6b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.496840 10257 tablet_replica.cc:333] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.497337 10257 raft_consensus.cc:2243] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.497707 10257 raft_consensus.cc:2272] T 017bc4b9de4c4462845e7b860bfa6dce P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.499558 10257 tablet_replica.cc:333] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.500098 10257 raft_consensus.cc:2243] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.500864 10257 raft_consensus.cc:2272] T d4cca6423ee4465cb73609e4e335e66e P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.502790 10257 tablet_replica.cc:333] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.503288 10257 raft_consensus.cc:2243] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.503749 10257 raft_consensus.cc:2272] T 0c3445b301b2410eba0473bd236b619c P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.506343 10257 tablet_replica.cc:333] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.507114 10257 raft_consensus.cc:2243] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.507614 10257 raft_consensus.cc:2272] T 6247cfa8622e49e29ad0bfba76525834 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.510360 10257 tablet_replica.cc:333] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.511055 10257 raft_consensus.cc:2243] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.511610 10257 raft_consensus.cc:2272] T fcb98cd681e0496393f8af9091c9184b P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.513288 18164 consensus_peers.cc:597] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.514155 10257 tablet_replica.cc:333] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.514776 10257 raft_consensus.cc:2243] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.515203 10257 raft_consensus.cc:2272] T 0aa2530e8b624041bc011fa928d7078f P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.517129 10257 tablet_replica.cc:333] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.517692 10257 raft_consensus.cc:2243] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.518433 10257 raft_consensus.cc:2272] T 27dcd1944f074b89a6dd10d75e8c6dd2 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.520256 10257 tablet_replica.cc:333] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.520758 10257 raft_consensus.cc:2243] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.521184 10257 raft_consensus.cc:2272] T 46d0d9f275f44ae59f4f6c2df48acd6d P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.523438 10257 tablet_replica.cc:333] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.524025 10257 raft_consensus.cc:2243] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.524487 10257 raft_consensus.cc:2272] T 843fde44d03641ce89e8d2e247c68b2d P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.527154 10257 tablet_replica.cc:333] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.527933 10257 raft_consensus.cc:2243] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.528856 10257 raft_consensus.cc:2272] T c9de6c54e6314854955dc74a83ea79d3 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.531126 10257 tablet_replica.cc:333] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.531780 10257 raft_consensus.cc:2243] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.532563 10257 raft_consensus.cc:2272] T 1f30e4471448411bab26f97e6d34f33c P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.534789 10257 tablet_replica.cc:333] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.535372 10257 raft_consensus.cc:2243] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.535915 10257 raft_consensus.cc:2272] T 9f4120b61f1d44f6a4a3c5c4575f993a P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.538064 10257 tablet_replica.cc:333] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.538614 10257 raft_consensus.cc:2243] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.539049 10257 raft_consensus.cc:2272] T a7241c7eca1b4c53a4faea0bcffee354 P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.541270 10257 tablet_replica.cc:333] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.541778 10257 raft_consensus.cc:2243] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.542198 10257 raft_consensus.cc:2272] T 6165069b9c0d4a2bb1637ab3ade3fe09 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.544399 10257 tablet_replica.cc:333] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.544901 10257 raft_consensus.cc:2243] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.545318 10257 raft_consensus.cc:2272] T eb77b44e31244ec29dad965bc258249a P fa03f4668c724e22a467d4e621f95272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.547937 10257 tablet_replica.cc:333] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272: stopping tablet replica
I20260709 13:18:21.548542 10257 raft_consensus.cc:2243] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.549502 10257 raft_consensus.cc:2272] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fa03f4668c724e22a467d4e621f95272 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.551270 18164 consensus_peers.cc:597] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.619688 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
W20260709 13:18:21.662149 18090 consensus_peers.cc:597] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.665391 18164 consensus_peers.cc:597] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.691897 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
W20260709 13:18:21.698551 18090 consensus_peers.cc:597] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.705890 18166 consensus_peers.cc:597] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.715283 18166 consensus_peers.cc:597] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.716320 18164 consensus_peers.cc:597] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.727445 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:21.728299 10257 tablet_replica.cc:333] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.729154 10257 raft_consensus.cc:2243] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.730199 10257 raft_consensus.cc:2272] T 4bd55fcdd2224c1db74729d68acc0ee6 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.732406 10257 tablet_replica.cc:333] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.732959 10257 raft_consensus.cc:2243] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.733345 10257 raft_consensus.cc:2272] T c143c20585e74a46807a580f6dcdafc1 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.735225 10257 tablet_replica.cc:333] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.735841 10257 raft_consensus.cc:2243] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.736222 10257 raft_consensus.cc:2272] T 79c18e4eea964029ad14dc02ce2137e2 P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.738034 10257 tablet_replica.cc:333] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.738520 10257 raft_consensus.cc:2243] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.738878 10257 raft_consensus.cc:2272] T b2861a57f9034f7fa50b7ff5bd98789b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.740723 10257 tablet_replica.cc:333] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.741218 10257 raft_consensus.cc:2243] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.741997 10257 raft_consensus.cc:2272] T 46d0d9f275f44ae59f4f6c2df48acd6d P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.743909 10257 tablet_replica.cc:333] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.744640 10257 raft_consensus.cc:2243] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.745430 10257 raft_consensus.cc:2272] T 020d6114c75749cfab1d9c0b047da996 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.747953 10257 tablet_replica.cc:333] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230: stopping tablet replica
W20260709 13:18:21.748400 18164 consensus_peers.cc:597] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.748715 10257 raft_consensus.cc:2243] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.749238 10257 raft_consensus.cc:2272] T f1ef84f551b343d3ad57cf6aa39cf47f P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.751116 10257 tablet_replica.cc:333] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.751726 10257 raft_consensus.cc:2243] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.752123 10257 raft_consensus.cc:2272] T b7077b3a2710489282f1c1cd3f8c1ddb P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.754488 10257 tablet_replica.cc:333] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230: stopping tablet replica
W20260709 13:18:21.754571 18166 consensus_peers.cc:597] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.755124 10257 raft_consensus.cc:2243] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.755632 10257 raft_consensus.cc:2272] T 0c3445b301b2410eba0473bd236b619c P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.757504 10257 tablet_replica.cc:333] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.758037 10257 raft_consensus.cc:2243] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.758395 10257 raft_consensus.cc:2272] T 8fee47e66a974e48926c551ee307b756 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.760377 10257 tablet_replica.cc:333] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.760906 10257 raft_consensus.cc:2243] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.761271 10257 raft_consensus.cc:2272] T eb3ec66980be4a6ea8be1b3fa67292b8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.763213 10257 tablet_replica.cc:333] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.763922 10257 raft_consensus.cc:2243] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.764384 10257 raft_consensus.cc:2272] T 15a02bb6e41942bba7f6cc5358ef1506 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.766247 18166 consensus_peers.cc:597] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.766636 10257 tablet_replica.cc:333] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.767328 10257 raft_consensus.cc:2243] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.767765 10257 raft_consensus.cc:2272] T 127bfde2c3624e5bb2d0f68929889394 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.769977 10257 tablet_replica.cc:333] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.770552 10257 raft_consensus.cc:2243] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.771311 10257 raft_consensus.cc:2272] T 691cfc20fdd444b3b53db6fdcdedcda8 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.773479 10257 tablet_replica.cc:333] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.774037 10257 raft_consensus.cc:2243] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.774796 10257 raft_consensus.cc:2272] T 30604e2cd10b450fb3e8d52bee20f8da P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.777117 10257 tablet_replica.cc:333] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.777871 10257 raft_consensus.cc:2243] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.778898 10257 raft_consensus.cc:2272] T 3918cfe707c24309bd3a731cca5bcbef P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.781209 10257 tablet_replica.cc:333] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.781785 10257 raft_consensus.cc:2243] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.782225 10257 raft_consensus.cc:2272] T 848c8b98e4dd44e398eb0b2689ea69cd P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.784557 10257 tablet_replica.cc:333] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.785306 10257 raft_consensus.cc:2243] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.785866 10257 raft_consensus.cc:2272] T 1f30e4471448411bab26f97e6d34f33c P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.788529 10257 tablet_replica.cc:333] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.789103 10257 raft_consensus.cc:2243] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.789945 10257 raft_consensus.cc:2272] T 00cc4ceb4dee449a8998229f3bd78288 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.792806 10257 tablet_replica.cc:333] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.793599 10257 raft_consensus.cc:2243] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.794127 10257 raft_consensus.cc:2272] T fcb98cd681e0496393f8af9091c9184b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.796851 10257 tablet_replica.cc:333] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.797601 10257 raft_consensus.cc:2243] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.798121 10257 raft_consensus.cc:2272] T bc6bc2204cfe4f09a8f0bae7240822d8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.800794 10257 tablet_replica.cc:333] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.801578 10257 raft_consensus.cc:2243] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.802095 10257 raft_consensus.cc:2272] T 27dcd1944f074b89a6dd10d75e8c6dd2 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.804256 10257 tablet_replica.cc:333] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.804778 10257 raft_consensus.cc:2243] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.805176 10257 raft_consensus.cc:2272] T da457bda0979490fa4e5277fba14ee84 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.807040 10257 tablet_replica.cc:333] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.807576 10257 raft_consensus.cc:2243] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.807948 10257 raft_consensus.cc:2272] T 1c67ac9ecbb841cea07be431d6a78d54 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.809719 10257 tablet_replica.cc:333] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.810267 10257 raft_consensus.cc:2243] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.810716 10257 raft_consensus.cc:2272] T d4cca6423ee4465cb73609e4e335e66e P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.812743 10257 tablet_replica.cc:333] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.813246 10257 raft_consensus.cc:2243] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.813621 10257 raft_consensus.cc:2272] T 3ad41548503545059d74bd2564c17b52 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.815519 10257 tablet_replica.cc:333] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.816005 10257 raft_consensus.cc:2243] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.816385 10257 raft_consensus.cc:2272] T c65f480c3ad843578f87c43aee4ecbc6 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.818084 10257 tablet_replica.cc:333] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.818568 10257 raft_consensus.cc:2243] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.819312 10257 raft_consensus.cc:2272] T 0044e0ba4e7548ea8cd3452eed875fa8 P 1787628aa7f6489497bb871588797230 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.821228 10257 tablet_replica.cc:333] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.821899 10257 raft_consensus.cc:2243] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.822374 10257 raft_consensus.cc:2272] T 017bc4b9de4c4462845e7b860bfa6dce P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.824467 10257 tablet_replica.cc:333] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.825030 10257 raft_consensus.cc:2243] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.825556 10257 raft_consensus.cc:2272] T 72bffd5492454e96ae4e10907b97381c P 1787628aa7f6489497bb871588797230 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.826136 18164 consensus_peers.cc:597] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.827912 10257 tablet_replica.cc:333] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.828504 10257 raft_consensus.cc:2243] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.829211 10257 raft_consensus.cc:2272] T 285967e1504c483dac972f54a3d59bfc P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.830968 10257 tablet_replica.cc:333] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.831526 10257 raft_consensus.cc:2243] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.831938 10257 raft_consensus.cc:2272] T 0745baef9dbd4cd0a943445a5aa9cc6b P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.833786 10257 tablet_replica.cc:333] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.834251 10257 raft_consensus.cc:2243] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.834633 10257 raft_consensus.cc:2272] T 6165069b9c0d4a2bb1637ab3ade3fe09 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.836555 10257 tablet_replica.cc:333] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.837021 10257 raft_consensus.cc:2243] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.837409 10257 raft_consensus.cc:2272] T eb77b44e31244ec29dad965bc258249a P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.839491 10257 tablet_replica.cc:333] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.840008 10257 raft_consensus.cc:2243] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.840385 10257 raft_consensus.cc:2272] T 7d07ebb0703849dbab22922f7f878958 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.842130 10257 tablet_replica.cc:333] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.842604 10257 raft_consensus.cc:2243] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.842974 10257 raft_consensus.cc:2272] T 729ea2fca9fb40e5a0c2fd994a5adb91 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.845134 10257 tablet_replica.cc:333] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.845777 10257 raft_consensus.cc:2243] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.846715 10257 raft_consensus.cc:2272] T 6200167e8cfc45a59b2b043a2601401f P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.849203 10257 tablet_replica.cc:333] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.849921 10257 raft_consensus.cc:2243] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.850769 10257 raft_consensus.cc:2272] T d97db5bc48fc437fac40454dc3e1d871 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.853309 10257 tablet_replica.cc:333] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.854069 10257 raft_consensus.cc:2243] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.854576 10257 raft_consensus.cc:2272] T 8989b7b66db644f89f6c4421fd51f2cc P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.856561 18164 consensus_peers.cc:597] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.856920 10257 tablet_replica.cc:333] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.857679 10257 raft_consensus.cc:2243] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.858682 10257 raft_consensus.cc:2272] T dd76d33e673a4966a9c91b362fafb0b1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.860555 18166 consensus_peers.cc:597] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.861016 10257 tablet_replica.cc:333] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.861564 10257 raft_consensus.cc:2243] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.862268 10257 raft_consensus.cc:2272] T 7187dfc4efc34078a81a1f59637521c8 P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.864416 10257 tablet_replica.cc:333] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.865037 10257 raft_consensus.cc:2243] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.865895 10257 raft_consensus.cc:2272] T f26de913e9ce4dad845f7173d82adfc1 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.867944 18166 consensus_peers.cc:597] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.868274 10257 tablet_replica.cc:333] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.868851 10257 raft_consensus.cc:2243] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:21.869313 10257 raft_consensus.cc:2272] T 9f4120b61f1d44f6a4a3c5c4575f993a P 1787628aa7f6489497bb871588797230 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:21.871855 10257 tablet_replica.cc:333] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230: stopping tablet replica
I20260709 13:18:21.872462 10257 raft_consensus.cc:2243] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:21.873425 10257 raft_consensus.cc:2272] T a7241c7eca1b4c53a4faea0bcffee354 P 1787628aa7f6489497bb871588797230 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:21.885399 18166 consensus_peers.cc:597] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.900096 18164 consensus_peers.cc:597] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 -> Peer fa03f4668c724e22a467d4e621f95272 (127.10.4.66:40273): Couldn't send request to peer fa03f4668c724e22a467d4e621f95272. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:40273: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:21.936072 18166 consensus_peers.cc:597] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 -> Peer 1787628aa7f6489497bb871588797230 (127.10.4.67:43399): Couldn't send request to peer 1787628aa7f6489497bb871588797230. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.67:43399: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:21.943306 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
I20260709 13:18:22.011704 10257 tablet_server.cc:179] TabletServer@127.10.4.68:0 shutting down...
I20260709 13:18:22.060120 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:22.061206 10257 tablet_replica.cc:333] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.062301 10257 raft_consensus.cc:2243] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.062978 10257 raft_consensus.cc:2272] T 110dca2dfd484809b9e058a7ab22b49a P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.065411 10257 tablet_replica.cc:333] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.066375 10257 raft_consensus.cc:2243] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.066964 10257 raft_consensus.cc:2272] T 1c67ac9ecbb841cea07be431d6a78d54 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.070003 10257 tablet_replica.cc:333] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.070916 10257 raft_consensus.cc:2243] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.071483 10257 raft_consensus.cc:2272] T b8f2c37906b94d31b85748436677deef P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.073617 10257 tablet_replica.cc:333] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.074203 10257 raft_consensus.cc:2243] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.075044 10257 raft_consensus.cc:2272] T c143c20585e74a46807a580f6dcdafc1 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.077172 10257 tablet_replica.cc:333] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.077766 10257 raft_consensus.cc:2243] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.078176 10257 raft_consensus.cc:2272] T 79c18e4eea964029ad14dc02ce2137e2 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.080331 10257 tablet_replica.cc:333] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.081151 10257 raft_consensus.cc:2243] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.081645 10257 raft_consensus.cc:2272] T b2861a57f9034f7fa50b7ff5bd98789b P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.084244 10257 tablet_replica.cc:333] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.084895 10257 raft_consensus.cc:2243] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.085295 10257 raft_consensus.cc:2272] T 15a02bb6e41942bba7f6cc5358ef1506 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.087456 10257 tablet_replica.cc:333] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.088058 10257 raft_consensus.cc:2243] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.088482 10257 raft_consensus.cc:2272] T f1ef84f551b343d3ad57cf6aa39cf47f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.090384 10257 tablet_replica.cc:333] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.090925 10257 raft_consensus.cc:2243] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.091709 10257 raft_consensus.cc:2272] T b7077b3a2710489282f1c1cd3f8c1ddb P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.093700 10257 tablet_replica.cc:333] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.094242 10257 raft_consensus.cc:2243] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.095022 10257 raft_consensus.cc:2272] T eb77b44e31244ec29dad965bc258249a P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.096881 10257 tablet_replica.cc:333] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.097401 10257 raft_consensus.cc:2243] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.097781 10257 raft_consensus.cc:2272] T 7d07ebb0703849dbab22922f7f878958 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.099618 10257 tablet_replica.cc:333] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.100102 10257 raft_consensus.cc:2243] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.100489 10257 raft_consensus.cc:2272] T 729ea2fca9fb40e5a0c2fd994a5adb91 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.102213 10257 tablet_replica.cc:333] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.102733 10257 raft_consensus.cc:2243] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.103087 10257 raft_consensus.cc:2272] T 0aa2530e8b624041bc011fa928d7078f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.104791 10257 tablet_replica.cc:333] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.105288 10257 raft_consensus.cc:2243] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.106012 10257 raft_consensus.cc:2272] T bc6bc2204cfe4f09a8f0bae7240822d8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.107923 10257 tablet_replica.cc:333] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.108436 10257 raft_consensus.cc:2243] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.108831 10257 raft_consensus.cc:2272] T 6200167e8cfc45a59b2b043a2601401f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.110569 10257 tablet_replica.cc:333] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.111081 10257 raft_consensus.cc:2243] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.111521 10257 raft_consensus.cc:2272] T d97db5bc48fc437fac40454dc3e1d871 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.113518 10257 tablet_replica.cc:333] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.114058 10257 raft_consensus.cc:2243] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.114832 10257 raft_consensus.cc:2272] T f6de10a3b7bb4a8da2902e5f42108ed2 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.116963 10257 tablet_replica.cc:333] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.117527 10257 raft_consensus.cc:2243] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.117947 10257 raft_consensus.cc:2272] T 347efb4b336e41469ded6211da2d4498 P fc8a9ec31a5745d69dae4262b271e568 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.120050 10257 tablet_replica.cc:333] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.120724 10257 raft_consensus.cc:2243] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.121685 10257 raft_consensus.cc:2272] T e1606bb8512a4c5ba5133be786298984 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.124171 10257 tablet_replica.cc:333] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.124823 10257 raft_consensus.cc:2243] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.125633 10257 raft_consensus.cc:2272] T a4c004aeb9af431087cd2e10becd16c2 P fc8a9ec31a5745d69dae4262b271e568 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.127547 10257 tablet_replica.cc:333] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.128093 10257 raft_consensus.cc:2243] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.128489 10257 raft_consensus.cc:2272] T 691cfc20fdd444b3b53db6fdcdedcda8 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.130378 10257 tablet_replica.cc:333] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.130909 10257 raft_consensus.cc:2243] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.131294 10257 raft_consensus.cc:2272] T 30604e2cd10b450fb3e8d52bee20f8da P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.133302 10257 tablet_replica.cc:333] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.133806 10257 raft_consensus.cc:2243] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.134203 10257 raft_consensus.cc:2272] T 3918cfe707c24309bd3a731cca5bcbef P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.135946 10257 tablet_replica.cc:333] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.136432 10257 raft_consensus.cc:2243] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.136799 10257 raft_consensus.cc:2272] T 848c8b98e4dd44e398eb0b2689ea69cd P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.138612 10257 tablet_replica.cc:333] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.139114 10257 raft_consensus.cc:2243] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.139521 10257 raft_consensus.cc:2272] T c65f480c3ad843578f87c43aee4ecbc6 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.141237 10257 tablet_replica.cc:333] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.141729 10257 raft_consensus.cc:2243] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.142108 10257 raft_consensus.cc:2272] T f26de913e9ce4dad845f7173d82adfc1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.143863 10257 tablet_replica.cc:333] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.144340 10257 raft_consensus.cc:2243] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.144706 10257 raft_consensus.cc:2272] T 3fd2d32e1713467387bb231dc639c10d P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.146420 10257 tablet_replica.cc:333] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.146970 10257 raft_consensus.cc:2243] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.147378 10257 raft_consensus.cc:2272] T 00cc4ceb4dee449a8998229f3bd78288 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.149194 10257 tablet_replica.cc:333] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.149741 10257 raft_consensus.cc:2243] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.150151 10257 raft_consensus.cc:2272] T 211547823b99438c9cc10432cc919a5f P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.152103 10257 tablet_replica.cc:333] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.152722 10257 raft_consensus.cc:2243] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.153173 10257 raft_consensus.cc:2272] T da457bda0979490fa4e5277fba14ee84 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.155277 10257 tablet_replica.cc:333] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.155853 10257 raft_consensus.cc:2243] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.156303 10257 raft_consensus.cc:2272] T f0771f9326ce4105929b0fca4a301e1e P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.158298 10257 tablet_replica.cc:333] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.158871 10257 raft_consensus.cc:2243] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.159302 10257 raft_consensus.cc:2272] T d4bd29ab75004ddf9b1ce255a071ed24 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.161183 10257 tablet_replica.cc:333] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.161728 10257 raft_consensus.cc:2243] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.162540 10257 raft_consensus.cc:2272] T 3ad41548503545059d74bd2564c17b52 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.164615 10257 tablet_replica.cc:333] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.165141 10257 raft_consensus.cc:2243] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.165585 10257 raft_consensus.cc:2272] T 7187dfc4efc34078a81a1f59637521c8 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.167519 10257 tablet_replica.cc:333] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.168013 10257 raft_consensus.cc:2243] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.168766 10257 raft_consensus.cc:2272] T 0c3445b301b2410eba0473bd236b619c P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.170480 10257 tablet_replica.cc:333] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.170958 10257 raft_consensus.cc:2243] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.171687 10257 raft_consensus.cc:2272] T 6247cfa8622e49e29ad0bfba76525834 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.173444 10257 tablet_replica.cc:333] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.173960 10257 raft_consensus.cc:2243] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.174685 10257 raft_consensus.cc:2272] T 72bffd5492454e96ae4e10907b97381c P fc8a9ec31a5745d69dae4262b271e568 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.176548 10257 tablet_replica.cc:333] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.177145 10257 raft_consensus.cc:2243] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.177623 10257 raft_consensus.cc:2272] T 285967e1504c483dac972f54a3d59bfc P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.179976 10257 tablet_replica.cc:333] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.180408 10257 raft_consensus.cc:2243] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.181097 10257 raft_consensus.cc:2272] T 0745baef9dbd4cd0a943445a5aa9cc6b P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.182832 10257 tablet_replica.cc:333] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.183264 10257 raft_consensus.cc:2243] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.183982 10257 raft_consensus.cc:2272] T 843fde44d03641ce89e8d2e247c68b2d P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.185654 10257 tablet_replica.cc:333] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.186097 10257 raft_consensus.cc:2243] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.186450 10257 raft_consensus.cc:2272] T c9de6c54e6314854955dc74a83ea79d3 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.188242 10257 tablet_replica.cc:333] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.188655 10257 raft_consensus.cc:2243] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.189003 10257 raft_consensus.cc:2272] T 1f30e4471448411bab26f97e6d34f33c P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.190788 10257 tablet_replica.cc:333] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.191183 10257 raft_consensus.cc:2243] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:22.191558 10257 raft_consensus.cc:2272] T dd76d33e673a4966a9c91b362fafb0b1 P fc8a9ec31a5745d69dae4262b271e568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.193311 10257 tablet_replica.cc:333] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568: stopping tablet replica
I20260709 13:18:22.193730 10257 raft_consensus.cc:2243] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.194437 10257 raft_consensus.cc:2272] T a80b08582e1a4e099492c0650e820c45 P fc8a9ec31a5745d69dae4262b271e568 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.258126 10257 tablet_server.cc:196] TabletServer@127.10.4.68:0 shutdown complete.
I20260709 13:18:22.316605 10257 master.cc:562] Master@127.10.4.126:39849 shutting down...
I20260709 13:18:22.337230 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:22.337731 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:22.337992 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P f30b4505e27e480386540645d141f06c: stopping tablet replica
I20260709 13:18:22.403141 10257 master.cc:584] Master@127.10.4.126:39849 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (42104 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:18:22.486891 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:34531
I20260709 13:18:22.488018 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:22.494004 19205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:22.494113 19204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:22.494674 10257 server_base.cc:1061] running on GCE node
W20260709 13:18:22.494194 19207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:22.496973 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:22.497151 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:22.497303 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603102497286 us; error 0 us; skew 500 ppm
I20260709 13:18:22.497784 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:22.499989 10257 webserver.cc:533] Webserver started at http://127.10.4.126:41145/ using document root <none> and password file <none>
I20260709 13:18:22.500434 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:22.500598 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:22.500833 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:22.501878 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "6f13b409408b4d53968cc4195eb6302c"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.505998 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:18:22.509290 19212 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.509966 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:18:22.510255 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "6f13b409408b4d53968cc4195eb6302c"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.510510 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:22.545226 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:22.546373 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:22.583992 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:34531
I20260709 13:18:22.584100 19263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:34531 every 8 connection(s)
I20260709 13:18:22.587673 19264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:22.597450 19264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c: Bootstrap starting.
I20260709 13:18:22.602110 19264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:22.606038 19264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c: No bootstrap required, opened a new log
I20260709 13:18:22.608162 19264 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f13b409408b4d53968cc4195eb6302c" member_type: VOTER }
I20260709 13:18:22.608523 19264 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:22.608731 19264 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f13b409408b4d53968cc4195eb6302c, State: Initialized, Role: FOLLOWER
I20260709 13:18:22.609298 19264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f13b409408b4d53968cc4195eb6302c" member_type: VOTER }
I20260709 13:18:22.609730 19264 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:18:22.609928 19264 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:18:22.610179 19264 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:22.614670 19264 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f13b409408b4d53968cc4195eb6302c" member_type: VOTER }
I20260709 13:18:22.615130 19264 leader_election.cc:304] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f13b409408b4d53968cc4195eb6302c; no voters: 
I20260709 13:18:22.616186 19264 leader_election.cc:290] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:18:22.616478 19267 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:22.617702 19267 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 1 LEADER]: Becoming Leader. State: Replica: 6f13b409408b4d53968cc4195eb6302c, State: Running, Role: LEADER
I20260709 13:18:22.618316 19267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f13b409408b4d53968cc4195eb6302c" member_type: VOTER }
I20260709 13:18:22.618862 19264 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:18:22.622202 19268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f13b409408b4d53968cc4195eb6302c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f13b409408b4d53968cc4195eb6302c" member_type: VOTER } }
I20260709 13:18:22.622828 19268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:22.623013 19269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f13b409408b4d53968cc4195eb6302c. Latest consensus state: current_term: 1 leader_uuid: "6f13b409408b4d53968cc4195eb6302c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f13b409408b4d53968cc4195eb6302c" member_type: VOTER } }
I20260709 13:18:22.623756 19269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:22.629577 19274 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:18:22.634337 19274 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:18:22.637157 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:18:22.642772 19274 catalog_manager.cc:1383] Generated new cluster ID: 49caba5ff1d041ae9d11e575ef1cfd3c
I20260709 13:18:22.643066 19274 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:18:22.657086 19274 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:18:22.658460 19274 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:18:22.669992 19274 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c: Generated new TSK 0
I20260709 13:18:22.670594 19274 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:18:22.704538 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:22.710294 19285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:22.711012 19286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:22.713533 10257 server_base.cc:1061] running on GCE node
W20260709 13:18:22.713832 19288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:22.714982 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:22.715178 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:22.715302 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603102715291 us; error 0 us; skew 500 ppm
I20260709 13:18:22.715837 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:22.717996 10257 webserver.cc:533] Webserver started at http://127.10.4.65:38525/ using document root <none> and password file <none>
I20260709 13:18:22.718416 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:22.718580 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:22.718791 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:22.719944 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "ef42340151804641b647fd666754cb24"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.724136 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:22.727233 19293 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.727939 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:22.728186 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "ef42340151804641b647fd666754cb24"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.728412 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:22.740550 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:22.741636 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:22.742980 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:22.745180 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:22.745359 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.745584 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:22.745743 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.784512 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:46655
I20260709 13:18:22.784682 19355 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:46655 every 8 connection(s)
I20260709 13:18:22.789494 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:22.796372 19360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:22.797228 19361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:22.799242 19363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:22.800824 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:22.801968 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:18:22.802024 19356 heartbeater.cc:344] Connected to a master server at 127.10.4.126:34531
W20260709 13:18:22.802198 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:22.802433 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603102802420 us; error 0 us; skew 500 ppm
I20260709 13:18:22.802495 19356 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:22.803010 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:22.803316 19356 heartbeater.cc:507] Master 127.10.4.126:34531 requested a full tablet report, sending...
I20260709 13:18:22.805680 19229 ts_manager.cc:194] Registered new tserver with Master: ef42340151804641b647fd666754cb24 (127.10.4.65:46655)
I20260709 13:18:22.805709 10257 webserver.cc:533] Webserver started at http://127.10.4.66:33729/ using document root <none> and password file <none>
I20260709 13:18:22.806367 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:22.806552 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:22.806859 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:22.807528 19229 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43530
I20260709 13:18:22.807956 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.812496 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:22.815819 19368 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.816540 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:18:22.816807 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.817070 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:22.829572 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:22.830724 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:22.832295 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:22.834476 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:22.834666 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.834882 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:22.835024 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.873929 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:34673
I20260709 13:18:22.874053 19430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:34673 every 8 connection(s)
I20260709 13:18:22.878436 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:22.885592 19434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:22.887472 19435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:22.892243 19431 heartbeater.cc:344] Connected to a master server at 127.10.4.126:34531
I20260709 13:18:22.892753 19431 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:22.893739 19431 heartbeater.cc:507] Master 127.10.4.126:34531 requested a full tablet report, sending...
I20260709 13:18:22.894500 10257 server_base.cc:1061] running on GCE node
W20260709 13:18:22.894790 19437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:22.895973 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:22.896238 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:22.896255 19229 ts_manager.cc:194] Registered new tserver with Master: 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:22.896507 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603102896484 us; error 0 us; skew 500 ppm
I20260709 13:18:22.897457 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:22.898262 19229 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43538
I20260709 13:18:22.900652 10257 webserver.cc:533] Webserver started at http://127.10.4.67:34089/ using document root <none> and password file <none>
I20260709 13:18:22.901203 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:22.901342 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:22.901525 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:22.902753 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "ca787ea0df384e469d634c565caed010"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.907867 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:18:22.911537 19442 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.912505 10257 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:18:22.912806 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "ca787ea0df384e469d634c565caed010"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:22.913147 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:22.933140 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:22.934319 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:22.935741 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:22.937888 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:22.938086 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.938305 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:22.938452 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:22.976136 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:39861
I20260709 13:18:22.976254 19505 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:39861 every 8 connection(s)
I20260709 13:18:22.988507 19506 heartbeater.cc:344] Connected to a master server at 127.10.4.126:34531
I20260709 13:18:22.988863 19506 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:22.989589 19506 heartbeater.cc:507] Master 127.10.4.126:34531 requested a full tablet report, sending...
I20260709 13:18:22.991453 19229 ts_manager.cc:194] Registered new tserver with Master: ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:22.992441 10257 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013116605s
I20260709 13:18:22.992815 19229 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43550
I20260709 13:18:23.017396 19229 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43564:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:23.020293 19229 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:23.094067 19321 tablet_service.cc:1511] Processing CreateTablet for tablet d605fd708e0e42ea8c5d9a117b1ddbff (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:18:23.095530 19321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d605fd708e0e42ea8c5d9a117b1ddbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.096668 19471 tablet_service.cc:1511] Processing CreateTablet for tablet d605fd708e0e42ea8c5d9a117b1ddbff (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:18:23.098033 19471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d605fd708e0e42ea8c5d9a117b1ddbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.100864 19470 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3f28ae190e415b8e6f445b64db75fa (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:18:23.102334 19470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3f28ae190e415b8e6f445b64db75fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.103892 19320 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3f28ae190e415b8e6f445b64db75fa (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:18:23.104331 19319 tablet_service.cc:1511] Processing CreateTablet for tablet 92874c9d488048f29020dca908fb0427 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:18:23.105422 19320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3f28ae190e415b8e6f445b64db75fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.106097 19319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92874c9d488048f29020dca908fb0427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.110598 19469 tablet_service.cc:1511] Processing CreateTablet for tablet 92874c9d488048f29020dca908fb0427 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:18:23.112121 19469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92874c9d488048f29020dca908fb0427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.112926 19318 tablet_service.cc:1511] Processing CreateTablet for tablet 72074f684d42435799b0418ad21fc105 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:18:23.113814 19466 tablet_service.cc:1511] Processing CreateTablet for tablet 52e3a1dbf8a54a39a92579d94298f8f0 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:18:23.114518 19318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72074f684d42435799b0418ad21fc105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.114936 19467 tablet_service.cc:1511] Processing CreateTablet for tablet 587a4db012a54f76a3a1889c1862a1e4 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:18:23.113818 19468 tablet_service.cc:1511] Processing CreateTablet for tablet 72074f684d42435799b0418ad21fc105 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:18:23.115263 19466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e3a1dbf8a54a39a92579d94298f8f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.118640 19468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72074f684d42435799b0418ad21fc105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.120906 19467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587a4db012a54f76a3a1889c1862a1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.124598 19317 tablet_service.cc:1511] Processing CreateTablet for tablet 587a4db012a54f76a3a1889c1862a1e4 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:18:23.126152 19317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587a4db012a54f76a3a1889c1862a1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.126377 19465 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed3c39b2d9a4413ad19c07d583a1582 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:18:23.127854 19465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed3c39b2d9a4413ad19c07d583a1582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.128919 19314 tablet_service.cc:1511] Processing CreateTablet for tablet 46e70c8fee654c7d8a3a28f64d48482e (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:18:23.130304 19314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e70c8fee654c7d8a3a28f64d48482e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.133189 19315 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed3c39b2d9a4413ad19c07d583a1582 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:18:23.134541 19315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed3c39b2d9a4413ad19c07d583a1582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.134871 19313 tablet_service.cc:1511] Processing CreateTablet for tablet 0a20484fdf86439b806bc0b08e492fa8 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:18:23.136184 19313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a20484fdf86439b806bc0b08e492fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.137385 19464 tablet_service.cc:1511] Processing CreateTablet for tablet 46e70c8fee654c7d8a3a28f64d48482e (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:18:23.127775 19316 tablet_service.cc:1511] Processing CreateTablet for tablet 52e3a1dbf8a54a39a92579d94298f8f0 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:18:23.139130 19464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e70c8fee654c7d8a3a28f64d48482e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.139791 19316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e3a1dbf8a54a39a92579d94298f8f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.142086 19396 tablet_service.cc:1511] Processing CreateTablet for tablet d605fd708e0e42ea8c5d9a117b1ddbff (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:18:23.144526 19395 tablet_service.cc:1511] Processing CreateTablet for tablet 3b3f28ae190e415b8e6f445b64db75fa (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:18:23.146780 19394 tablet_service.cc:1511] Processing CreateTablet for tablet 92874c9d488048f29020dca908fb0427 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:18:23.149031 19393 tablet_service.cc:1511] Processing CreateTablet for tablet 72074f684d42435799b0418ad21fc105 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:18:23.151149 19392 tablet_service.cc:1511] Processing CreateTablet for tablet 587a4db012a54f76a3a1889c1862a1e4 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:18:23.152853 19463 tablet_service.cc:1511] Processing CreateTablet for tablet 0a20484fdf86439b806bc0b08e492fa8 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:18:23.154910 19463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a20484fdf86439b806bc0b08e492fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.155328 19396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d605fd708e0e42ea8c5d9a117b1ddbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.154093 19391 tablet_service.cc:1511] Processing CreateTablet for tablet 52e3a1dbf8a54a39a92579d94298f8f0 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:18:23.156080 19395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b3f28ae190e415b8e6f445b64db75fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.156675 19393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72074f684d42435799b0418ad21fc105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.155234 19389 tablet_service.cc:1511] Processing CreateTablet for tablet 46e70c8fee654c7d8a3a28f64d48482e (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:18:23.157281 19392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 587a4db012a54f76a3a1889c1862a1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.158412 19394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92874c9d488048f29020dca908fb0427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.155234 19390 tablet_service.cc:1511] Processing CreateTablet for tablet 6ed3c39b2d9a4413ad19c07d583a1582 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:18:23.160845 19462 tablet_service.cc:1511] Processing CreateTablet for tablet c2afe64a363b452fbac62c22914151fd (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:18:23.161414 19390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed3c39b2d9a4413ad19c07d583a1582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.162156 19462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2afe64a363b452fbac62c22914151fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.163758 19391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e3a1dbf8a54a39a92579d94298f8f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.164883 19389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e70c8fee654c7d8a3a28f64d48482e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.176664 19312 tablet_service.cc:1511] Processing CreateTablet for tablet c2afe64a363b452fbac62c22914151fd (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:18:23.178088 19312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2afe64a363b452fbac62c22914151fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.212575 19388 tablet_service.cc:1511] Processing CreateTablet for tablet 0a20484fdf86439b806bc0b08e492fa8 (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:18:23.213994 19388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a20484fdf86439b806bc0b08e492fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.219579 19387 tablet_service.cc:1511] Processing CreateTablet for tablet c2afe64a363b452fbac62c22914151fd (DEFAULT_TABLE table=test-workload [id=81fe9aa878b444d8b623ef9d3e3223d8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:18:23.221057 19387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2afe64a363b452fbac62c22914151fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:23.344918 19522 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.344918 19523 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.353824 19523 tablet_bootstrap.cc:654] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.354801 19522 tablet_bootstrap.cc:654] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.363935 19524 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.370924 19523 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.371598 19523 ts_tablet_manager.cc:1403] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.013s
I20260709 13:18:23.375550 19524 tablet_bootstrap.cc:654] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.376842 19523 raft_consensus.cc:359] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.378297 19523 raft_consensus.cc:385] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.378683 19523 raft_consensus.cc:740] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.379886 19522 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.380007 19523 consensus_queue.cc:260] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.382725 19522 ts_tablet_manager.cc:1403] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.013s
I20260709 13:18:23.383853 19356 heartbeater.cc:499] Master 127.10.4.126:34531 was elected leader, sending a full tablet report...
I20260709 13:18:23.384357 19523 ts_tablet_manager.cc:1434] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.002s
I20260709 13:18:23.386013 19523 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.386876 19522 raft_consensus.cc:359] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.389750 19522 raft_consensus.cc:385] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.390079 19522 raft_consensus.cc:740] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.391490 19524 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.391084 19522 consensus_queue.cc:260] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.392046 19524 ts_tablet_manager.cc:1403] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.012s
I20260709 13:18:23.393461 19523 tablet_bootstrap.cc:654] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.395753 19524 raft_consensus.cc:359] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.396725 19524 raft_consensus.cc:385] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.397101 19524 raft_consensus.cc:740] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.398305 19524 consensus_queue.cc:260] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.401044 19522 ts_tablet_manager.cc:1434] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.003s
I20260709 13:18:23.402352 19506 heartbeater.cc:499] Master 127.10.4.126:34531 was elected leader, sending a full tablet report...
I20260709 13:18:23.406210 19523 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.406852 19523 ts_tablet_manager.cc:1403] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:18:23.409981 19523 raft_consensus.cc:359] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.410684 19523 raft_consensus.cc:385] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.410995 19523 raft_consensus.cc:740] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.412009 19523 consensus_queue.cc:260] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.414216 19523 ts_tablet_manager.cc:1434] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:18:23.415263 19523 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.416198 19522 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.417887 19431 heartbeater.cc:499] Master 127.10.4.126:34531 was elected leader, sending a full tablet report...
I20260709 13:18:23.418327 19524 ts_tablet_manager.cc:1434] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.026s	user 0.022s	sys 0.004s
I20260709 13:18:23.420033 19524 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.421540 19523 tablet_bootstrap.cc:654] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.426980 19524 tablet_bootstrap.cc:654] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.438403 19522 tablet_bootstrap.cc:654] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.439132 19524 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.439698 19524 ts_tablet_manager.cc:1403] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.007s
I20260709 13:18:23.442829 19524 raft_consensus.cc:359] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.443627 19524 raft_consensus.cc:385] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.443982 19523 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.443933 19524 raft_consensus.cc:740] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.444584 19523 ts_tablet_manager.cc:1403] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.013s
I20260709 13:18:23.445102 19524 consensus_queue.cc:260] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.447450 19524 ts_tablet_manager.cc:1434] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:18:23.447607 19522 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.448148 19522 ts_tablet_manager.cc:1403] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.008s
I20260709 13:18:23.447963 19523 raft_consensus.cc:359] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.448590 19524 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.448817 19523 raft_consensus.cc:385] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.449210 19523 raft_consensus.cc:740] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.450143 19523 consensus_queue.cc:260] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.451572 19522 raft_consensus.cc:359] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.452363 19522 raft_consensus.cc:385] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.452442 19523 ts_tablet_manager.cc:1434] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:18:23.452705 19522 raft_consensus.cc:740] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.455260 19523 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.456411 19524 tablet_bootstrap.cc:654] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.463126 19523 tablet_bootstrap.cc:654] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.463675 19522 consensus_queue.cc:260] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.470465 19524 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.471076 19524 ts_tablet_manager.cc:1403] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.008s
I20260709 13:18:23.474298 19524 raft_consensus.cc:359] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.474825 19523 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.475154 19524 raft_consensus.cc:385] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.475288 19522 ts_tablet_manager.cc:1434] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:18:23.475368 19523 ts_tablet_manager.cc:1403] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:18:23.475559 19524 raft_consensus.cc:740] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.476761 19524 consensus_queue.cc:260] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.478058 19523 raft_consensus.cc:359] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.478749 19523 raft_consensus.cc:385] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.479043 19523 raft_consensus.cc:740] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.479357 19524 ts_tablet_manager.cc:1434] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:18:23.479425 19522 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.479867 19523 consensus_queue.cc:260] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.480690 19524 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
W20260709 13:18:23.481752 19507 tablet.cc:2406] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:18:23.482555 19523 ts_tablet_manager.cc:1434] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:23.483696 19523 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.487840 19524 tablet_bootstrap.cc:654] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.489645 19522 tablet_bootstrap.cc:654] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.490483 19523 tablet_bootstrap.cc:654] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.498577 19522 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.499183 19522 ts_tablet_manager.cc:1403] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.005s
I20260709 13:18:23.500982 19529 raft_consensus.cc:493] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.501675 19529 raft_consensus.cc:515] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.504611 19523 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.505308 19523 ts_tablet_manager.cc:1403] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.012s
I20260709 13:18:23.508836 19523 raft_consensus.cc:359] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.510349 19523 raft_consensus.cc:385] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.510713 19523 raft_consensus.cc:740] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.511616 19523 consensus_queue.cc:260] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.514391 19523 ts_tablet_manager.cc:1434] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:18:23.515730 19523 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.517565 19529 leader_election.cc:290] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:23.523783 19531 raft_consensus.cc:493] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.524408 19531 raft_consensus.cc:515] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.525031 19522 raft_consensus.cc:359] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.525969 19522 raft_consensus.cc:385] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.526296 19522 raft_consensus.cc:740] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.527674 19531 leader_election.cc:290] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.523137 19523 tablet_bootstrap.cc:654] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.540534 19524 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.541136 19524 ts_tablet_manager.cc:1403] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.061s	user 0.020s	sys 0.024s
I20260709 13:18:23.544540 19524 raft_consensus.cc:359] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.545336 19524 raft_consensus.cc:385] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.545621 19524 raft_consensus.cc:740] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.546243 19522 consensus_queue.cc:260] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.546365 19524 consensus_queue.cc:260] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.554927 19528 raft_consensus.cc:493] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.555689 19528 raft_consensus.cc:515] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.562539 19528 leader_election.cc:290] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.555049 19524 ts_tablet_manager.cc:1434] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 13:18:23.555013 19522 ts_tablet_manager.cc:1434] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 13:18:23.573648 19524 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.578094 19522 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.581681 19524 tablet_bootstrap.cc:654] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.583364 19528 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.583969 19528 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.586438 19528 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.593726 19524 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.594492 19524 ts_tablet_manager.cc:1403] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.002s
I20260709 13:18:23.594635 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
W20260709 13:18:23.596575 19443 leader_election.cc:343] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.598248 19524 raft_consensus.cc:359] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.599068 19524 raft_consensus.cc:385] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.599421 19524 raft_consensus.cc:740] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.600657 19523 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.601226 19523 ts_tablet_manager.cc:1403] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.086s	user 0.033s	sys 0.029s
I20260709 13:18:23.601980 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:23.603261 19331 raft_consensus.cc:2468] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 0.
I20260709 13:18:23.600251 19524 consensus_queue.cc:260] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.605484 19523 raft_consensus.cc:359] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.606267 19523 raft_consensus.cc:385] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.606544 19523 raft_consensus.cc:740] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.607182 19524 ts_tablet_manager.cc:1434] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:18:23.607421 19523 consensus_queue.cc:260] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.605101 19445 leader_election.cc:304] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:23.608464 19524 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.609153 19529 raft_consensus.cc:2804] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:23.609573 19529 raft_consensus.cc:493] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.609915 19529 raft_consensus.cc:3060] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.610404 19523 ts_tablet_manager.cc:1434] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:18:23.617254 19524 tablet_bootstrap.cc:654] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.625635 19523 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.631654 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
W20260709 13:18:23.633256 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:23.633806 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
W20260709 13:18:23.635665 19295 leader_election.cc:343] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:23.636363 19295 leader_election.cc:343] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.642676 19529 raft_consensus.cc:515] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.648947 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:23.649044 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:23.649792 19330 raft_consensus.cc:3060] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:23.650744 19443 leader_election.cc:343] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.651494 19529 leader_election.cc:290] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:23.644379 19523 tablet_bootstrap.cc:654] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.633659 19522 tablet_bootstrap.cc:654] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.644395 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:23.655360 19331 raft_consensus.cc:2468] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 0.
I20260709 13:18:23.655781 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:23.657536 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:23.657698 19371 leader_election.cc:304] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:23.658406 19405 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 0.
I20260709 13:18:23.658784 19531 raft_consensus.cc:2804] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:23.659237 19531 raft_consensus.cc:493] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.659623 19531 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:23.660818 19294 leader_election.cc:343] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.661489 19294 leader_election.cc:304] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef42340151804641b647fd666754cb24; no voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010
I20260709 13:18:23.663015 19294 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: ca787ea0df384e469d634c565caed010
I20260709 13:18:23.664156 19528 raft_consensus.cc:2749] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:23.664983 19528 raft_consensus.cc:2804] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:23.665341 19528 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.665659 19528 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.673265 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:23.674368 19531 raft_consensus.cc:515] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
W20260709 13:18:23.675309 19370 leader_election.cc:343] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.678202 19524 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.678799 19524 ts_tablet_manager.cc:1403] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.071s	user 0.022s	sys 0.033s
I20260709 13:18:23.679494 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:23.680203 19331 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.682253 19524 raft_consensus.cc:359] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.683065 19524 raft_consensus.cc:385] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.683598 19524 raft_consensus.cc:740] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.684540 19524 consensus_queue.cc:260] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.682461 19528 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.688802 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:23.690055 19331 raft_consensus.cc:2468] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 1.
W20260709 13:18:23.690922 19295 leader_election.cc:343] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.691296 19371 leader_election.cc:304] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:23.693503 19528 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Requested vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.697727 19330 raft_consensus.cc:2468] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:23.699158 19445 leader_election.cc:304] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:23.700654 19529 raft_consensus.cc:2804] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:23.700909 19524 ts_tablet_manager.cc:1434] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.022s	user 0.015s	sys 0.006s
I20260709 13:18:23.702239 19524 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.709033 19524 tablet_bootstrap.cc:654] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.716137 19555 raft_consensus.cc:2804] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:23.717140 19294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.66:34673, user_credentials={real_user=slave}} blocked reactor thread for 54224.4us
I20260709 13:18:23.718608 19529 raft_consensus.cc:697] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:23.719499 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:23.720150 19405 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.719841 19529 consensus_queue.cc:237] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.723232 19524 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.723893 19524 ts_tablet_manager.cc:1403] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:18:23.726313 19522 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.726462 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:23.726902 19522 ts_tablet_manager.cc:1403] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.149s	user 0.022s	sys 0.026s
I20260709 13:18:23.727113 19524 raft_consensus.cc:359] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.727986 19524 raft_consensus.cc:385] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:18:23.728204 19370 leader_election.cc:343] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.728348 19524 raft_consensus.cc:740] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.729595 19524 consensus_queue.cc:260] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.730271 19522 raft_consensus.cc:359] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.731051 19522 raft_consensus.cc:385] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.731377 19522 raft_consensus.cc:740] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.732370 19524 ts_tablet_manager.cc:1434] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:18:23.732985 19405 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 1.
I20260709 13:18:23.728178 19555 raft_consensus.cc:697] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:23.734457 19294 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: ca787ea0df384e469d634c565caed010
I20260709 13:18:23.737430 19524 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.738602 19553 raft_consensus.cc:2804] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:23.737668 19522 consensus_queue.cc:260] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.739130 19553 raft_consensus.cc:697] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: Becoming Leader. State: Replica: ef42340151804641b647fd666754cb24, State: Running, Role: LEADER
I20260709 13:18:23.740042 19553 consensus_queue.cc:237] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.734467 19523 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.741297 19523 ts_tablet_manager.cc:1403] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.130s	user 0.019s	sys 0.024s
I20260709 13:18:23.740958 19229 catalog_manager.cc:5697] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.726495 19531 leader_election.cc:290] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.744555 19523 raft_consensus.cc:359] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.745321 19523 raft_consensus.cc:385] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.745672 19523 raft_consensus.cc:740] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.745841 19524 tablet_bootstrap.cc:654] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.740690 19522 ts_tablet_manager.cc:1434] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.006s
I20260709 13:18:23.746150 19555 consensus_queue.cc:237] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.747800 19523 consensus_queue.cc:260] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.750340 19523 ts_tablet_manager.cc:1434] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:18:23.750717 19529 raft_consensus.cc:493] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.751227 19529 raft_consensus.cc:515] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.753667 19522 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.754230 19523 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.756673 19228 catalog_manager.cc:5697] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef42340151804641b647fd666754cb24 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "ef42340151804641b647fd666754cb24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.759876 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20484fdf86439b806bc0b08e492fa8" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:23.761343 19522 tablet_bootstrap.cc:654] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:23.761554 19445 leader_election.cc:343] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.763608 19529 leader_election.cc:290] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:23.764396 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20484fdf86439b806bc0b08e492fa8" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:23.765216 19405 raft_consensus.cc:2468] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 0.
I20260709 13:18:23.765049 19523 tablet_bootstrap.cc:654] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.766647 19443 leader_election.cc:304] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: ef42340151804641b647fd666754cb24
I20260709 13:18:23.767989 19529 raft_consensus.cc:2804] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:23.768388 19529 raft_consensus.cc:493] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.768734 19529 raft_consensus.cc:3060] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.768925 19524 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.769513 19524 ts_tablet_manager.cc:1403] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.011s
I20260709 13:18:23.772519 19524 raft_consensus.cc:359] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.773237 19522 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.773794 19522 ts_tablet_manager.cc:1403] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.005s
I20260709 13:18:23.775869 19531 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.776434 19531 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.776800 19522 raft_consensus.cc:359] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.777117 19529 raft_consensus.cc:515] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.777580 19522 raft_consensus.cc:385] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.777894 19522 raft_consensus.cc:740] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.773216 19524 raft_consensus.cc:385] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.779086 19524 raft_consensus.cc:740] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.779523 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:23.780023 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:23.778681 19531 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
W20260709 13:18:23.781018 19371 leader_election.cc:343] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.778613 19522 consensus_queue.cc:260] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
W20260709 13:18:23.781878 19370 leader_election.cc:343] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.782384 19370 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4; no voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24
I20260709 13:18:23.779835 19524 consensus_queue.cc:260] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.783668 19558 raft_consensus.cc:493] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.783860 19522 ts_tablet_manager.cc:1434] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:18:23.784588 19558 raft_consensus.cc:515] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.785077 19524 ts_tablet_manager.cc:1434] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:18:23.785142 19522 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.786386 19524 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
I20260709 13:18:23.787479 19558 leader_election.cc:290] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.789197 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
W20260709 13:18:23.791054 19370 leader_election.cc:343] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.783108 19531 raft_consensus.cc:2749] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:23.792819 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:23.793166 19523 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.793208 19522 tablet_bootstrap.cc:654] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.793577 19331 raft_consensus.cc:2468] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 0.
I20260709 13:18:23.794283 19523 ts_tablet_manager.cc:1403] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.012s
I20260709 13:18:23.795046 19371 leader_election.cc:304] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: ca787ea0df384e469d634c565caed010
I20260709 13:18:23.795910 19524 tablet_bootstrap.cc:654] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.796290 19531 raft_consensus.cc:2804] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:23.796720 19531 raft_consensus.cc:493] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.797075 19531 raft_consensus.cc:3060] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.803726 19524 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.804222 19524 ts_tablet_manager.cc:1403] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:18:23.807279 19524 raft_consensus.cc:359] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.808032 19524 raft_consensus.cc:385] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.808368 19524 raft_consensus.cc:740] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.807668 19531 raft_consensus.cc:515] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.809660 19524 consensus_queue.cc:260] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.812361 19523 raft_consensus.cc:359] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.813149 19523 raft_consensus.cc:385] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.813457 19523 raft_consensus.cc:740] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.813463 19524 ts_tablet_manager.cc:1434] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:18:23.813979 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20484fdf86439b806bc0b08e492fa8" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:23.814585 19523 consensus_queue.cc:260] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.815055 19524 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4: Bootstrap starting.
W20260709 13:18:23.815539 19445 leader_election.cc:343] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.815881 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20484fdf86439b806bc0b08e492fa8" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:23.816460 19405 raft_consensus.cc:3060] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.817502 19523 ts_tablet_manager.cc:1434] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 13:18:23.818672 19523 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.819278 19556 raft_consensus.cc:493] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.819818 19556 raft_consensus.cc:515] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.821826 19556 leader_election.cc:290] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:23.822661 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3f28ae190e415b8e6f445b64db75fa" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:23.823006 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3f28ae190e415b8e6f445b64db75fa" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:23.823380 19331 raft_consensus.cc:2468] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 0.
I20260709 13:18:23.824216 19524 tablet_bootstrap.cc:654] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:23.824611 19443 leader_election.cc:343] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.825244 19445 leader_election.cc:304] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:23.826493 19556 raft_consensus.cc:2804] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:23.826867 19556 raft_consensus.cc:493] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.827057 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:23.827345 19556 raft_consensus.cc:3060] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.827381 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:23.827785 19331 raft_consensus.cc:3060] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:23.829134 19370 leader_election.cc:343] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.829694 19529 leader_election.cc:290] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:23.825326 19405 raft_consensus.cc:2468] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:23.831326 19443 leader_election.cc:304] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: ef42340151804641b647fd666754cb24
I20260709 13:18:23.832216 19531 leader_election.cc:290] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.835582 19531 raft_consensus.cc:493] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.836470 19531 raft_consensus.cc:515] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:23.838424 19523 tablet_bootstrap.cc:654] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.839453 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:23.840972 19556 raft_consensus.cc:515] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.842587 19531 leader_election.cc:290] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca787ea0df384e469d634c565caed010 (127.10.4.67:39861), ef42340151804641b647fd666754cb24 (127.10.4.65:46655)
I20260709 13:18:23.838295 19522 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.842846 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:23.843544 19522 ts_tablet_manager.cc:1403] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.008s
I20260709 13:18:23.843808 19330 raft_consensus.cc:2393] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93c3a69dd97f44f481ff44b3b667f3b4 in current term 1: Already voted for candidate ca787ea0df384e469d634c565caed010 in this term.
I20260709 13:18:23.843210 19529 raft_consensus.cc:2804] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:23.845587 19371 leader_election.cc:304] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4; no voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24
I20260709 13:18:23.842727 19331 raft_consensus.cc:2468] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 1.
I20260709 13:18:23.846465 19555 raft_consensus.cc:3060] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.847657 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3f28ae190e415b8e6f445b64db75fa" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:23.848294 19330 raft_consensus.cc:3060] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.850487 19226 catalog_manager.cc:5697] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.852070 19371 leader_election.cc:304] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: ca787ea0df384e469d634c565caed010
I20260709 13:18:23.852918 19558 raft_consensus.cc:2804] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:23.853389 19558 raft_consensus.cc:697] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:23.853761 19555 raft_consensus.cc:2749] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:23.854193 19558 consensus_queue.cc:237] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.857080 19523 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.857558 19523 ts_tablet_manager.cc:1403] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.002s
I20260709 13:18:23.858031 19330 raft_consensus.cc:2468] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:23.859513 19445 leader_election.cc:304] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:23.864037 19523 raft_consensus.cc:359] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.864846 19523 raft_consensus.cc:385] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.865137 19523 raft_consensus.cc:740] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.868754 19529 raft_consensus.cc:697] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:23.870798 19529 consensus_queue.cc:237] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.873288 19522 raft_consensus.cc:359] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.874810 19523 consensus_queue.cc:260] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.874467 19522 raft_consensus.cc:385] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.878290 19522 raft_consensus.cc:740] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.877051 19572 raft_consensus.cc:2804] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:23.882761 19524 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4: No bootstrap required, opened a new log
I20260709 13:18:23.886159 19556 leader_election.cc:290] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:23.886883 19524 ts_tablet_manager.cc:1403] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent bootstrapping tablet: real 0.069s	user 0.008s	sys 0.025s
I20260709 13:18:23.888299 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b3f28ae190e415b8e6f445b64db75fa" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:23.888991 19523 ts_tablet_manager.cc:1434] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:18:23.890735 19523 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:23.888978 19553 raft_consensus.cc:493] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.900200 19553 raft_consensus.cc:515] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.902184 19524 raft_consensus.cc:359] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.903004 19524 raft_consensus.cc:385] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.903261 19524 raft_consensus.cc:740] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.904268 19524 consensus_queue.cc:260] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.905458 19405 raft_consensus.cc:3060] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.907543 19522 consensus_queue.cc:260] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.907132 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:23.911098 19524 ts_tablet_manager.cc:1434] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:18:23.910449 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:23.908758 19572 raft_consensus.cc:697] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:23.916162 19406 raft_consensus.cc:2468] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 0.
W20260709 13:18:23.916357 19295 leader_election.cc:343] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.917624 19294 leader_election.cc:304] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: ca787ea0df384e469d634c565caed010
I20260709 13:18:23.916188 19572 consensus_queue.cc:237] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.918459 19528 raft_consensus.cc:2804] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:23.918825 19528 raft_consensus.cc:493] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.919140 19528 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.916146 19522 ts_tablet_manager.cc:1434] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 13:18:23.921404 19523 tablet_bootstrap.cc:654] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.909682 19553 leader_election.cc:290] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.926616 19528 raft_consensus.cc:515] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.931456 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:23.932085 19406 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:23.932459 19528 leader_election.cc:290] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Requested vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:23.933884 19405 raft_consensus.cc:2468] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:23.935206 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:23.939800 19523 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: No bootstrap required, opened a new log
I20260709 13:18:23.940344 19523 ts_tablet_manager.cc:1403] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.007s
W20260709 13:18:23.941238 19295 leader_election.cc:343] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca787ea0df384e469d634c565caed010 (127.10.4.67:39861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:23.943153 19523 raft_consensus.cc:359] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.943892 19523 raft_consensus.cc:385] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.941183 19228 catalog_manager.cc:5697] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.944144 19523 raft_consensus.cc:740] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.945334 19523 consensus_queue.cc:260] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.947688 19523 ts_tablet_manager.cc:1434] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:23.948227 19406 raft_consensus.cc:2468] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 1.
I20260709 13:18:23.949570 19294 leader_election.cc:304] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: ca787ea0df384e469d634c565caed010
I20260709 13:18:23.950683 19528 raft_consensus.cc:2804] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:23.950616 19522 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.951228 19528 raft_consensus.cc:697] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: Becoming Leader. State: Replica: ef42340151804641b647fd666754cb24, State: Running, Role: LEADER
I20260709 13:18:23.952158 19528 consensus_queue.cc:237] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.960553 19522 tablet_bootstrap.cc:654] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.958184 19226 catalog_manager.cc:5697] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.968710 19522 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.969203 19522 ts_tablet_manager.cc:1403] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.005s
I20260709 13:18:23.973237 19522 raft_consensus.cc:359] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.973974 19522 raft_consensus.cc:385] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.974280 19522 raft_consensus.cc:740] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.974977 19522 consensus_queue.cc:260] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.974980 19229 catalog_manager.cc:5697] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef42340151804641b647fd666754cb24 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "ef42340151804641b647fd666754cb24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.974831 19228 catalog_manager.cc:5697] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.977603 19522 ts_tablet_manager.cc:1434] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:18:23.978658 19522 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.984031 19522 tablet_bootstrap.cc:654] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:23.988508 19522 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:23.988952 19522 ts_tablet_manager.cc:1403] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:18:23.991063 19522 raft_consensus.cc:359] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.991545 19522 raft_consensus.cc:385] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:23.991732 19522 raft_consensus.cc:740] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:23.992285 19522 consensus_queue.cc:260] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:23.993784 19522 ts_tablet_manager.cc:1434] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:23.994481 19522 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:23.998487 19522 tablet_bootstrap.cc:654] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:24.002293 19522 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:24.002635 19522 ts_tablet_manager.cc:1403] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:24.004477 19522 raft_consensus.cc:359] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.004913 19522 raft_consensus.cc:385] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:24.005095 19522 raft_consensus.cc:740] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:24.005612 19522 consensus_queue.cc:260] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.007129 19522 ts_tablet_manager.cc:1434] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:18:24.007951 19522 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010: Bootstrap starting.
I20260709 13:18:24.012076 19522 tablet_bootstrap.cc:654] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:24.016317 19522 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010: No bootstrap required, opened a new log
I20260709 13:18:24.016664 19522 ts_tablet_manager.cc:1403] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010: Time spent bootstrapping tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:18:24.018497 19522 raft_consensus.cc:359] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.019006 19522 raft_consensus.cc:385] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:24.019192 19522 raft_consensus.cc:740] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Initialized, Role: FOLLOWER
I20260709 13:18:24.019739 19522 consensus_queue.cc:260] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.021211 19522 ts_tablet_manager.cc:1434] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:24.023538 19572 raft_consensus.cc:493] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.024081 19572 raft_consensus.cc:515] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.026150 19572 leader_election.cc:290] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:24.026852 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:24.027114 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:24.027462 19330 raft_consensus.cc:2468] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 0.
I20260709 13:18:24.027832 19406 raft_consensus.cc:2468] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 0.
I20260709 13:18:24.028676 19445 leader_election.cc:304] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:24.029296 19572 raft_consensus.cc:2804] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:24.029639 19572 raft_consensus.cc:493] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.029867 19572 raft_consensus.cc:3060] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.035171 19572 raft_consensus.cc:515] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.036738 19572 leader_election.cc:290] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:24.037067 19558 raft_consensus.cc:493] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.037686 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:24.037667 19558 raft_consensus.cc:515] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.038408 19330 raft_consensus.cc:3060] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.038818 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:24.039507 19406 raft_consensus.cc:3060] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.040135 19558 leader_election.cc:290] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:24.041831 19331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:24.042501 19331 raft_consensus.cc:2468] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 0.
I20260709 13:18:24.043772 19371 leader_election.cc:304] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:24.044555 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:24.045495 19528 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.045696 19481 raft_consensus.cc:2468] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 0.
I20260709 13:18:24.045979 19528 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.047422 19406 raft_consensus.cc:2468] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:24.048169 19528 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:24.048424 19558 raft_consensus.cc:2804] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:24.048609 19443 leader_election.cc:304] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: 
I20260709 13:18:24.048915 19558 raft_consensus.cc:493] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.048830 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:24.049505 19558 raft_consensus.cc:3060] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.049773 19572 raft_consensus.cc:2804] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:24.049656 19405 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 0.
I20260709 13:18:24.050482 19572 raft_consensus.cc:697] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:24.051230 19294 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:24.049606 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:24.051303 19572 consensus_queue.cc:237] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.052224 19528 raft_consensus.cc:2804] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:24.052263 19481 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 0.
I20260709 13:18:24.052711 19528 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.053128 19528 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.054396 19330 raft_consensus.cc:2468] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:24.057235 19558 raft_consensus.cc:515] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.060207 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:24.060801 19330 raft_consensus.cc:3060] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.060866 19558 leader_election.cc:290] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:24.061365 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:24.061714 19528 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.061954 19481 raft_consensus.cc:3060] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.061908 19226 catalog_manager.cc:5697] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:24.064035 19528 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Requested vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:24.065320 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:24.065713 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:24.066286 19405 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.066763 19480 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.067588 19330 raft_consensus.cc:2468] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 1.
I20260709 13:18:24.068845 19371 leader_election.cc:304] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:24.069686 19572 raft_consensus.cc:493] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.070194 19572 raft_consensus.cc:515] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.069716 19558 raft_consensus.cc:2804] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:24.072000 19558 raft_consensus.cc:697] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:24.072903 19405 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 1.
I20260709 13:18:24.072777 19558 consensus_queue.cc:237] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.073410 19572 leader_election.cc:290] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:24.072904 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:24.074108 19294 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:24.074625 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:24.075174 19528 raft_consensus.cc:2804] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:24.075713 19528 raft_consensus.cc:697] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: Becoming Leader. State: Replica: ef42340151804641b647fd666754cb24, State: Running, Role: LEADER
I20260709 13:18:24.076491 19528 consensus_queue.cc:237] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.077709 19481 raft_consensus.cc:2468] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 1.
I20260709 13:18:24.079907 19480 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef42340151804641b647fd666754cb24 in term 1.
I20260709 13:18:24.075357 19406 raft_consensus.cc:2393] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca787ea0df384e469d634c565caed010 in current term 1: Already voted for candidate ef42340151804641b647fd666754cb24 in this term.
I20260709 13:18:24.085728 19443 leader_election.cc:304] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca787ea0df384e469d634c565caed010; no voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24
I20260709 13:18:24.086941 19572 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.090971 19226 catalog_manager.cc:5697] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:24.091749 19228 catalog_manager.cc:5697] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef42340151804641b647fd666754cb24 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "ef42340151804641b647fd666754cb24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:24.094348 19572 raft_consensus.cc:2749] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:24.131445 19572 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.132010 19572 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.133750 19572 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:24.134493 19330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef42340151804641b647fd666754cb24" is_pre_election: true
I20260709 13:18:24.134706 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:24.135421 19406 raft_consensus.cc:2393] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca787ea0df384e469d634c565caed010 in current term 1: Already voted for candidate ef42340151804641b647fd666754cb24 in this term.
I20260709 13:18:24.136236 19443 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca787ea0df384e469d634c565caed010; no voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24
I20260709 13:18:24.136847 19572 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.141917 19529 consensus_queue.cc:1048] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:24.142935 19572 raft_consensus.cc:2749] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:24.153147 19528 consensus_queue.cc:1048] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:24.156813 19572 consensus_queue.cc:1048] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:18:24.169230 19528 consensus_queue.cc:1048] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:24.170255 19572 raft_consensus.cc:493] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.170799 19572 raft_consensus.cc:515] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.171234 10257 ts_manager.cc:295] Set tserver state for ef42340151804641b647fd666754cb24 to MAINTENANCE_MODE
I20260709 13:18:24.172475 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
I20260709 13:18:24.174268 19572 leader_election.cc:290] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:24.175813 19404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
W20260709 13:18:24.212826 19445 leader_election.cc:336] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:24.213429 19445 leader_election.cc:304] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca787ea0df384e469d634c565caed010; no voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24
I20260709 13:18:24.214268 19572 raft_consensus.cc:2749] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:24.222234 19591 raft_consensus.cc:493] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.222741 19591 raft_consensus.cc:515] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:24.225648 19591 leader_election.cc:290] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:24.227211 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20484fdf86439b806bc0b08e492fa8" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:24.228272 19405 raft_consensus.cc:2393] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef42340151804641b647fd666754cb24 in current term 1: Already voted for candidate ca787ea0df384e469d634c565caed010 in this term.
I20260709 13:18:24.227607 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20484fdf86439b806bc0b08e492fa8" candidate_uuid: "ef42340151804641b647fd666754cb24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:24.230918 19295 leader_election.cc:304] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef42340151804641b647fd666754cb24; no voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010
I20260709 13:18:24.231835 19591 raft_consensus.cc:3060] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.235807 19480 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.239908 19591 raft_consensus.cc:2749] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:24.244127 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:24.245287 10257 tablet_replica.cc:333] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.246090 10257 raft_consensus.cc:2243] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.246569 10257 raft_consensus.cc:2272] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.247809 19558 consensus_queue.cc:1048] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:24.249094 10257 tablet_replica.cc:333] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.249881 10257 raft_consensus.cc:2243] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.250329 10257 raft_consensus.cc:2272] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.252658 10257 tablet_replica.cc:333] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.253243 10257 raft_consensus.cc:2243] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.253647 10257 raft_consensus.cc:2272] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.256565 10257 tablet_replica.cc:333] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.257335 10257 raft_consensus.cc:2243] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.258078 10257 pending_rounds.cc:70] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Trying to abort 1 pending ops.
W20260709 13:18:24.261487 19371 consensus_peers.cc:597] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.262512 10257 pending_rounds.cc:77] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305638314312617984 op_type: NO_OP noop_request { }
I20260709 13:18:24.263041 10257 raft_consensus.cc:2889] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:24.263320 10257 raft_consensus.cc:2272] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.265723 10257 tablet_replica.cc:333] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.266274 10257 raft_consensus.cc:2243] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.266671 10257 raft_consensus.cc:2272] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.269028 10257 tablet_replica.cc:333] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.269579 10257 raft_consensus.cc:2243] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.269891 10257 raft_consensus.cc:2272] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.271461 10257 tablet_replica.cc:333] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.271915 10257 raft_consensus.cc:2243] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.272614 10257 raft_consensus.cc:2272] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.274626 10257 tablet_replica.cc:333] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.275105 10257 raft_consensus.cc:2243] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.275457 10257 raft_consensus.cc:2272] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.277029 10257 tablet_replica.cc:333] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.277447 10257 raft_consensus.cc:2243] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:24.277942 10257 pending_rounds.cc:70] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Trying to abort 1 pending ops.
I20260709 13:18:24.278097 10257 pending_rounds.cc:77] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305638313796268032 op_type: NO_OP noop_request { }
I20260709 13:18:24.278308 10257 raft_consensus.cc:2889] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:24.278515 10257 raft_consensus.cc:2272] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.280439 10257 tablet_replica.cc:333] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: stopping tablet replica
I20260709 13:18:24.280946 10257 raft_consensus.cc:2243] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:24.281288 10257 raft_consensus.cc:2272] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:24.309624 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
I20260709 13:18:24.330787 19572 consensus_queue.cc:1048] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:24.345448 19445 consensus_peers.cc:597] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.360531 19445 consensus_peers.cc:597] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.362918 19371 consensus_peers.cc:597] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:24.369028 19480 raft_consensus.cc:3060] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:24.375156 19558 consensus_queue.cc:1048] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:24.526371 19558 consensus_queue.cc:1048] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:24.534431 19529 consensus_queue.cc:1048] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:24.543725 19371 consensus_peers.cc:597] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.623904 19445 consensus_peers.cc:597] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:24.634404 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:24.637483 19529 consensus_queue.cc:1048] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:24.757531 19445 consensus_peers.cc:597] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:25.596699 19556 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.597157 19556 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.598733 19556 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:25.599798 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:25.600479 19405 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:25.601524 19443 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: 
I20260709 13:18:25.602286 19556 raft_consensus.cc:2804] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:25.602435 19445 leader_election.cc:336] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:25.602696 19556 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.603042 19556 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.607919 19556 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.609462 19556 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:25.610621 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:25.611249 19405 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:25.612967 19445 leader_election.cc:336] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:25.616891 19405 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:25.617851 19443 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: ef42340151804641b647fd666754cb24
I20260709 13:18:25.618618 19556 raft_consensus.cc:2804] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:25.619081 19556 raft_consensus.cc:697] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:25.619728 19556 consensus_queue.cc:237] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.627166 19226 catalog_manager.cc:5697] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 1 to 2, leader changed from ef42340151804641b647fd666754cb24 (127.10.4.65) to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:25.635799 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:25.673728 19555 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef42340151804641b647fd666754cb24)
I20260709 13:18:25.674896 19574 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef42340151804641b647fd666754cb24)
I20260709 13:18:25.674677 19555 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.675432 19574 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.676990 19555 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:25.677444 19574 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:25.678093 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010" is_pre_election: true
I20260709 13:18:25.678701 19480 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 1.
I20260709 13:18:25.679359 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:25.679939 19370 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: 
I20260709 13:18:25.679983 19405 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:25.681497 19555 raft_consensus.cc:2804] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:25.681843 19555 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef42340151804641b647fd666754cb24)
I20260709 13:18:25.682164 19555 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:25.682385 19445 leader_election.cc:336] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
W20260709 13:18:25.682637 19371 leader_election.cc:336] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:25.682972 19443 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: ef42340151804641b647fd666754cb24
I20260709 13:18:25.683768 19574 raft_consensus.cc:2804] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:25.684103 19574 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef42340151804641b647fd666754cb24)
I20260709 13:18:25.684432 19574 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.688838 19555 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.690805 19555 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:25.690984 19574 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.691912 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:25.692607 19480 raft_consensus.cc:2393] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 93c3a69dd97f44f481ff44b3b667f3b4 in current term 2: Already voted for candidate ca787ea0df384e469d634c565caed010 in this term.
I20260709 13:18:25.693498 19574 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
W20260709 13:18:25.694694 19371 leader_election.cc:336] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:25.694694 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:25.695385 19371 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4; no voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24
I20260709 13:18:25.695715 19405 raft_consensus.cc:2393] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ca787ea0df384e469d634c565caed010 in current term 2: Already voted for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in this term.
W20260709 13:18:25.696017 19445 leader_election.cc:336] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:25.696164 19555 raft_consensus.cc:2749] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:25.696803 19443 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca787ea0df384e469d634c565caed010; no voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24
I20260709 13:18:25.697372 19574 raft_consensus.cc:2749] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:25.789475 19574 raft_consensus.cc:493] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.789920 19574 raft_consensus.cc:515] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.791518 19574 leader_election.cc:290] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:25.792529 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:25.793089 19405 raft_consensus.cc:2468] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 1.
I20260709 13:18:25.793937 19443 leader_election.cc:304] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: 
I20260709 13:18:25.794714 19574 raft_consensus.cc:2804] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:25.794895 19445 leader_election.cc:336] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:25.794991 19574 raft_consensus.cc:493] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.795486 19574 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.800180 19574 raft_consensus.cc:515] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.801646 19574 leader_election.cc:290] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:25.802666 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:25.803239 19405 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:25.805023 19445 leader_election.cc:336] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:25.808187 19405 raft_consensus.cc:2468] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:25.809067 19443 leader_election.cc:304] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: ef42340151804641b647fd666754cb24
I20260709 13:18:25.809649 19574 raft_consensus.cc:2804] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:25.810003 19574 raft_consensus.cc:697] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:25.810612 19574 consensus_queue.cc:237] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:25.817935 19226 catalog_manager.cc:5697] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 1 to 2, leader changed from ef42340151804641b647fd666754cb24 (127.10.4.65) to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:26.038092 19445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111) [suppressed 111 similar messages]
W20260709 13:18:26.145157 19445 consensus_peers.cc:597] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:26.189177 19405 raft_consensus.cc:1275] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:18:26.190735 19574 consensus_queue.cc:1048] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:26.230433 19445 consensus_peers.cc:597] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:26.295771 19405 raft_consensus.cc:1275] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:18:26.297034 19574 consensus_queue.cc:1048] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:26.636690 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:18:26.796046 19445 consensus_peers.cc:597] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:26.867072 19371 consensus_peers.cc:597] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:26.971480 19445 consensus_peers.cc:597] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:26.979928 19371 consensus_peers.cc:597] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:27.007719 19445 consensus_peers.cc:597] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:27.036625 19371 consensus_peers.cc:597] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:27.152139 19445 consensus_peers.cc:597] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:18:27.484460 19556 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.484915 19556 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:27.486656 19556 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:27.487788 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" is_pre_election: true
I20260709 13:18:27.488425 19405 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:27.489537 19443 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: 
W20260709 13:18:27.489985 19445 leader_election.cc:336] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:27.490146 19556 raft_consensus.cc:2804] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:27.490581 19556 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.490881 19556 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:27.496662 19556 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:27.498334 19556 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 3 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:27.499439 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:27.500097 19405 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:27.501574 19445 leader_election.cc:336] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111)
I20260709 13:18:27.505514 19405 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 3.
I20260709 13:18:27.506541 19443 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ca787ea0df384e469d634c565caed010; no voters: ef42340151804641b647fd666754cb24
I20260709 13:18:27.507252 19556 raft_consensus.cc:2804] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:27.507676 19556 raft_consensus.cc:697] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:27.508448 19556 consensus_queue.cc:237] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:27.515372 19229 catalog_manager.cc:5697] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 1 to 3, leader changed from ef42340151804641b647fd666754cb24 (127.10.4.65) to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:27.637706 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:27.901757 19405 raft_consensus.cc:1275] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:27.903013 19556 consensus_queue.cc:1048] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:27.916623 19445 consensus_peers.cc:597] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:28.638676 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:18:28.683450 19445 consensus_peers.cc:597] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:28.768476 19445 consensus_peers.cc:597] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:18:28.972345 19371 consensus_peers.cc:597] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:18:29.331956 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:29.340215 19632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:29.340209 19629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:29.341279 19630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:29.342865 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:29.344060 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:29.344317 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:29.344534 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603109344512 us; error 0 us; skew 500 ppm
I20260709 13:18:29.345110 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:29.348109 10257 webserver.cc:533] Webserver started at http://127.10.4.65:38525/ using document root <none> and password file <none>
I20260709 13:18:29.348548 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:29.348709 10257 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:18:29.352312 10257 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:18:29.355942 19637 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:29.356803 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:18:29.357131 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "ef42340151804641b647fd666754cb24"
format_stamp: "Formatted at 2026-07-09 13:18:22 on dist-test-slave-gkm9"
I20260709 13:18:29.357484 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:29.374171 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:29.375377 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:29.376902 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:29.380568 19644 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:18:29.385236 19371 consensus_peers.cc:597] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:29.391851 19445 consensus_peers.cc:597] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:18:29.417891 19445 consensus_peers.cc:597] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:18:29.434613 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:18:29.434983 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.055s	user 0.002s	sys 0.000s
I20260709 13:18:29.435294 10257 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:18:29.439965 19644 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.455771 19644 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:18:29.456832 19644 tablet_bootstrap.cc:492] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.457520 19644 ts_tablet_manager.cc:1403] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 13:18:29.460347 19644 raft_consensus.cc:359] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.461176 19644 raft_consensus.cc:740] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.461886 19644 consensus_queue.cc:260] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.465312 19644 ts_tablet_manager.cc:1434] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.003s
I20260709 13:18:29.466157 19644 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Bootstrap starting.
W20260709 13:18:29.467234 19371 consensus_peers.cc:597] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:18:29.480427 19644 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.481225 19644 tablet_bootstrap.cc:492] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.481747 19644 ts_tablet_manager.cc:1403] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.001s
I20260709 13:18:29.482441 10257 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:18:29.482708 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.047s	user 0.044s	sys 0.000s
I20260709 13:18:29.484068 19644 raft_consensus.cc:359] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.484635 19644 raft_consensus.cc:740] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.485198 19644 consensus_queue.cc:260] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.486680 19644 ts_tablet_manager.cc:1434] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:29.487316 19644 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.498494 19644 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.499331 19644 tablet_bootstrap.cc:492] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.500022 19644 ts_tablet_manager.cc:1403] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:18:29.502089 19644 raft_consensus.cc:359] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.502637 19644 raft_consensus.cc:740] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.503289 19644 consensus_queue.cc:260] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.505010 19644 ts_tablet_manager.cc:1434] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:29.505697 19644 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.515009 19644 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.515764 19644 tablet_bootstrap.cc:492] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.516465 19644 ts_tablet_manager.cc:1403] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:18:29.518424 19644 raft_consensus.cc:359] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.519065 19644 raft_consensus.cc:740] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.519704 19644 consensus_queue.cc:260] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.521222 19644 ts_tablet_manager.cc:1434] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:29.521905 19644 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.533495 19644 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:18:29.534245 19644 tablet_bootstrap.cc:492] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.534773 19644 ts_tablet_manager.cc:1403] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:18:29.536358 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:46655
I20260709 13:18:29.536512 19706 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:46655 every 8 connection(s)
I20260709 13:18:29.537148 19644 raft_consensus.cc:359] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.537819 19644 raft_consensus.cc:740] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.538388 19644 consensus_queue.cc:260] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.540071 19644 ts_tablet_manager.cc:1434] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:18:29.540931 19644 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.548744 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:29.553565 19644 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.554503 19644 tablet_bootstrap.cc:492] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.555258 19644 ts_tablet_manager.cc:1403] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:18:29.557991 19644 raft_consensus.cc:359] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.558779 19644 raft_consensus.cc:740] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.559535 19644 consensus_queue.cc:260] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.561987 19644 ts_tablet_manager.cc:1434] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:29.562973 19644 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.565295 19707 heartbeater.cc:344] Connected to a master server at 127.10.4.126:34531
I20260709 13:18:29.565783 19707 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:29.566793 19707 heartbeater.cc:507] Master 127.10.4.126:34531 requested a full tablet report, sending...
I20260709 13:18:29.573653 19229 ts_manager.cc:194] Re-registered known tserver with Master: ef42340151804641b647fd666754cb24 (127.10.4.65:46655)
I20260709 13:18:29.579277 19681 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:29.579958 19480 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4"
dest_uuid: "ca787ea0df384e469d634c565caed010"
mode: GRACEFUL
new_leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:40510
I20260709 13:18:29.580716 19480 raft_consensus.cc:606] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 LEADER]: Received request to transfer leadership to TS 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:29.582320 19644 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.583285 19644 tablet_bootstrap.cc:492] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.584029 19644 ts_tablet_manager.cc:1403] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.004s
I20260709 13:18:29.584017 19480 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582"
dest_uuid: "ca787ea0df384e469d634c565caed010"
mode: GRACEFUL
new_leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:40510
I20260709 13:18:29.584738 19480 raft_consensus.cc:606] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Received request to transfer leadership to TS 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:29.586804 19229 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45684
I20260709 13:18:29.586810 19644 raft_consensus.cc:359] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:29.587819 19644 raft_consensus.cc:740] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.588121 19480 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92874c9d488048f29020dca908fb0427"
dest_uuid: "ca787ea0df384e469d634c565caed010"
mode: GRACEFUL
new_leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:40510
I20260709 13:18:29.588847 19480 raft_consensus.cc:606] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Received request to transfer leadership to TS 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:29.588613 19644 consensus_queue.cc:260] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:29.590775 19707 heartbeater.cc:499] Master 127.10.4.126:34531 was elected leader, sending a full tablet report...
I20260709 13:18:29.590803 19644 ts_tablet_manager.cc:1434] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:29.590771 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:18:29.592115 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:29.592759 19644 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.592916 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:29.604862 19644 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.605543 19644 tablet_bootstrap.cc:492] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.606076 19644 ts_tablet_manager.cc:1403] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:18:29.607906 19644 raft_consensus.cc:359] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.608371 19644 raft_consensus.cc:740] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.608995 19644 consensus_queue.cc:260] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.610546 19644 ts_tablet_manager.cc:1434] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:29.611299 19644 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Bootstrap starting.
I20260709 13:18:29.621902 19644 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.622653 19644 tablet_bootstrap.cc:492] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.623238 19644 ts_tablet_manager.cc:1403] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:18:29.625182 19644 raft_consensus.cc:359] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.625810 19644 raft_consensus.cc:740] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.626451 19644 consensus_queue.cc:260] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.628621 19644 ts_tablet_manager.cc:1434] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:29.629223 19644 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Bootstrap starting.
W20260709 13:18:29.639593 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:29.640446 19644 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:29.641136 19644 tablet_bootstrap.cc:492] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Bootstrap complete.
I20260709 13:18:29.641649 19644 ts_tablet_manager.cc:1403] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:18:29.643617 19644 raft_consensus.cc:359] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.644097 19644 raft_consensus.cc:740] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef42340151804641b647fd666754cb24, State: Initialized, Role: FOLLOWER
I20260709 13:18:29.644618 19644 consensus_queue.cc:260] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.645959 19644 ts_tablet_manager.cc:1434] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:18:29.772737 19681 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.778527 19681 pending_rounds.cc:85] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Aborting all ops after (but not including) 0
I20260709 13:18:29.778767 19681 pending_rounds.cc:107] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:18:29.779084 19681 raft_consensus.cc:2889] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:29.813894 19681 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.818854 19681 pending_rounds.cc:85] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Aborting all ops after (but not including) 0
I20260709 13:18:29.819067 19681 pending_rounds.cc:107] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:18:29.819276 19681 raft_consensus.cc:2889] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:29.903785 19724 raft_consensus.cc:993] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010: : Instructing follower 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:29.904265 19626 raft_consensus.cc:1081] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 LEADER]: Signalling peer 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:29.905827 19405 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:35156
I20260709 13:18:29.906311 19405 raft_consensus.cc:493] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:29.906692 19405 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:29.912802 19405 raft_consensus.cc:515] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.914242 19724 raft_consensus.cc:993] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010: : Instructing follower 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:29.914853 19724 raft_consensus.cc:1081] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Signalling peer 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:29.915951 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:29.915958 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "587a4db012a54f76a3a1889c1862a1e4" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:29.916531 19406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:35156
I20260709 13:18:29.916805 19681 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:29.914968 19405 leader_election.cc:290] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 4 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:29.916829 19480 raft_consensus.cc:3055] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:29.917505 19406 raft_consensus.cc:493] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:29.917503 19480 raft_consensus.cc:740] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:29.917999 19406 raft_consensus.cc:3060] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.918525 19480 consensus_queue.cc:260] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.919749 19480 raft_consensus.cc:3060] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:29.924178 19681 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 4.
I20260709 13:18:29.925233 19371 leader_election.cc:304] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:29.925832 19555 raft_consensus.cc:2804] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:29.926105 19406 raft_consensus.cc:515] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.926883 19555 raft_consensus.cc:697] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:29.927145 19480 raft_consensus.cc:2468] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 4.
I20260709 13:18:29.927744 19555 consensus_queue.cc:237] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.928620 19406 leader_election.cc:290] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:29.929315 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:29.929634 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ed3c39b2d9a4413ad19c07d583a1582" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:29.930025 19681 raft_consensus.cc:3060] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.930249 19480 raft_consensus.cc:3055] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:29.930714 19480 raft_consensus.cc:740] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:29.931618 19480 consensus_queue.cc:260] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.933027 19480 raft_consensus.cc:3060] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:29.935671 19681 raft_consensus.cc:2468] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 2.
I20260709 13:18:29.936703 19371 leader_election.cc:304] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:29.937338 19728 raft_consensus.cc:2804] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:29.941129 19480 raft_consensus.cc:2468] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 2.
I20260709 13:18:29.945055 19728 raft_consensus.cc:697] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:29.945988 19728 consensus_queue.cc:237] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.947438 19229 catalog_manager.cc:5697] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 3 to 4, leader changed from ca787ea0df384e469d634c565caed010 (127.10.4.67) to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 4 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:29.964797 19226 catalog_manager.cc:5697] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 1 to 2, leader changed from ca787ea0df384e469d634c565caed010 (127.10.4.67) to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:29.988600 19626 raft_consensus.cc:993] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: : Instructing follower 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:29.989073 19724 raft_consensus.cc:1081] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Signalling peer 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:29.990581 19406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92874c9d488048f29020dca908fb0427"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:35156
I20260709 13:18:29.991073 19406 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:29.991349 19406 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:29.996528 19406 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:29.998247 19406 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 3 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:29.998700 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:29.999294 19681 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:29.999274 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:29.999998 19480 raft_consensus.cc:3055] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:30.000247 19480 raft_consensus.cc:740] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:30.000871 19480 consensus_queue.cc:260] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.001788 19480 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:30.004472 19681 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 3.
I20260709 13:18:30.005439 19371 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:30.006153 19728 raft_consensus.cc:2804] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:30.006505 19728 raft_consensus.cc:697] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:30.006729 19480 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 3.
I20260709 13:18:30.007236 19728 consensus_queue.cc:237] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.013690 19226 catalog_manager.cc:5697] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 2 to 3, leader changed from ca787ea0df384e469d634c565caed010 (127.10.4.67) to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:30.398195 19681 raft_consensus.cc:1275] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:30.399276 19728 consensus_queue.cc:1048] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.408648 19480 raft_consensus.cc:1275] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:30.410210 19730 consensus_queue.cc:1048] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.475967 19480 raft_consensus.cc:1275] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:30.477111 19730 consensus_queue.cc:1048] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.481020 19681 raft_consensus.cc:1275] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 4 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:30.483150 19730 consensus_queue.cc:1048] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:30.490582 19681 raft_consensus.cc:1275] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:30.495321 19480 raft_consensus.cc:1275] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 4 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:30.497077 19744 consensus_queue.cc:1048] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:30.497457 19730 consensus_queue.cc:1048] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:30.602808 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:30.619204 19406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
mode: GRACEFUL
new_leader_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:56026
I20260709 13:18:30.619774 19406 raft_consensus.cc:606] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Received request to transfer leadership to TS ca787ea0df384e469d634c565caed010
I20260709 13:18:30.622092 19406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
mode: GRACEFUL
new_leader_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:56026
I20260709 13:18:30.622606 19406 raft_consensus.cc:606] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Received request to transfer leadership to TS ca787ea0df384e469d634c565caed010
I20260709 13:18:30.623679 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:30.624146 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:30.624835 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:30.627643 19730 raft_consensus.cc:993] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4: : Instructing follower ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:30.628093 19744 raft_consensus.cc:1081] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Signalling peer ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:30.629561 19480 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e"
dest_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:18:30.630125 19480 raft_consensus.cc:493] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:30.630479 19480 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.636133 19480 raft_consensus.cc:515] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.637681 19480 leader_election.cc:290] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:30.638481 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:30.638737 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:30.639159 19681 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.639328 19406 raft_consensus.cc:3055] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:30.639748 19406 raft_consensus.cc:740] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
W20260709 13:18:30.640484 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:30.640522 19406 consensus_queue.cc:260] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.641916 19406 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.646443 19681 raft_consensus.cc:2468] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:30.647603 19445 leader_election.cc:304] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:30.648229 19626 raft_consensus.cc:2804] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:30.648249 19406 raft_consensus.cc:2468] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:30.648805 19626 raft_consensus.cc:697] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:30.649590 19626 consensus_queue.cc:237] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.656526 19228 catalog_manager.cc:5697] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 1 to 2, leader changed from 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66) to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:30.837512 19730 raft_consensus.cc:993] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4: : Instructing follower ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:30.837908 19744 raft_consensus.cc:1081] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Signalling peer ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:30.839188 19480 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd"
dest_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:18:30.839653 19480 raft_consensus.cc:493] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:30.839933 19480 raft_consensus.cc:3060] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.844563 19480 raft_consensus.cc:515] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.845980 19480 leader_election.cc:290] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:30.846688 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:30.846944 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2afe64a363b452fbac62c22914151fd" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:30.847220 19681 raft_consensus.cc:3060] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.847525 19406 raft_consensus.cc:3055] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:30.847796 19406 raft_consensus.cc:740] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:30.848425 19406 consensus_queue.cc:260] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.849318 19406 raft_consensus.cc:3060] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:30.852330 19681 raft_consensus.cc:2468] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:30.853269 19445 leader_election.cc:304] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:30.853868 19626 raft_consensus.cc:2804] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:30.854257 19626 raft_consensus.cc:697] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:30.854719 19406 raft_consensus.cc:2468] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:30.855021 19626 consensus_queue.cc:237] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:30.861967 19228 catalog_manager.cc:5697] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 1 to 2, leader changed from 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66) to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.113044 19681 raft_consensus.cc:1275] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:31.114323 19626 consensus_queue.cc:1048] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.124079 19406 raft_consensus.cc:1275] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:31.125927 19739 consensus_queue.cc:1048] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.324028 19681 raft_consensus.cc:1275] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:31.325239 19756 consensus_queue.cc:1048] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.335660 19406 raft_consensus.cc:1275] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:31.336800 19739 consensus_queue.cc:1048] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.633442 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:31.640343 19480 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e"
dest_uuid: "ca787ea0df384e469d634c565caed010"
mode: GRACEFUL
new_leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:40510
I20260709 13:18:31.640825 19480 raft_consensus.cc:606] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Received request to transfer leadership to TS 93c3a69dd97f44f481ff44b3b667f3b4
W20260709 13:18:31.641415 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:31.643362 19480 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72074f684d42435799b0418ad21fc105"
dest_uuid: "ca787ea0df384e469d634c565caed010"
mode: GRACEFUL
new_leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:40510
I20260709 13:18:31.643904 19480 raft_consensus.cc:606] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Received request to transfer leadership to TS 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:31.644874 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:31.645342 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:31.645901 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:31.678016 19739 raft_consensus.cc:993] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010: : Instructing follower 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:31.678532 19756 raft_consensus.cc:1081] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Signalling peer 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:31.680042 19406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:35156
I20260709 13:18:31.680540 19406 raft_consensus.cc:493] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.680886 19406 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.685587 19406 raft_consensus.cc:515] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:31.687036 19406 leader_election.cc:290] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 3 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:31.687757 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:31.687975 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46e70c8fee654c7d8a3a28f64d48482e" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:31.688380 19681 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.688468 19480 raft_consensus.cc:3055] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:31.688827 19480 raft_consensus.cc:740] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:31.689388 19480 consensus_queue.cc:260] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:31.690193 19480 raft_consensus.cc:3060] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.693081 19681 raft_consensus.cc:2468] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 3.
I20260709 13:18:31.694320 19371 leader_election.cc:304] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:31.694396 19480 raft_consensus.cc:2468] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 3.
I20260709 13:18:31.694954 19744 raft_consensus.cc:2804] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:31.696035 19744 raft_consensus.cc:697] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:31.696847 19744 consensus_queue.cc:237] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:31.702518 19228 catalog_manager.cc:5697] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 2 to 3, leader changed from ca787ea0df384e469d634c565caed010 (127.10.4.67) to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:31.993938 19739 raft_consensus.cc:993] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010: : Instructing follower 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:31.994385 19756 raft_consensus.cc:1081] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Signalling peer 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:31.995877 19406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72074f684d42435799b0418ad21fc105"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:35156
I20260709 13:18:31.996377 19406 raft_consensus.cc:493] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.996657 19406 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.001559 19406 raft_consensus.cc:515] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.003295 19406 leader_election.cc:290] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 3 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:32.004077 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:32.004344 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72074f684d42435799b0418ad21fc105" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:32.004690 19681 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.004860 19480 raft_consensus.cc:3055] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:32.005133 19480 raft_consensus.cc:740] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:32.005689 19480 consensus_queue.cc:260] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.006506 19480 raft_consensus.cc:3060] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.009519 19681 raft_consensus.cc:2468] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 3.
I20260709 13:18:32.010627 19371 leader_election.cc:304] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:32.011267 19744 raft_consensus.cc:2804] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:32.011714 19744 raft_consensus.cc:697] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:32.012609 19480 raft_consensus.cc:2468] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 3.
I20260709 13:18:32.012508 19744 consensus_queue.cc:237] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.019306 19228 catalog_manager.cc:5697] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 2 to 3, leader changed from ca787ea0df384e469d634c565caed010 (127.10.4.67) to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 3 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.119530 19681 raft_consensus.cc:1275] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.121029 19744 consensus_queue.cc:1048] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.134286 19480 raft_consensus.cc:1275] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.135587 19762 consensus_queue.cc:1048] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.523034 19681 raft_consensus.cc:1275] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:32.524195 19762 consensus_queue.cc:1048] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.531381 19480 raft_consensus.cc:1275] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:32.533252 19762 consensus_queue.cc:1048] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:32.642493 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:32.652241 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:32.659471 19406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
mode: GRACEFUL
new_leader_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:56026
I20260709 13:18:32.660004 19406 raft_consensus.cc:606] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Received request to transfer leadership to TS ca787ea0df384e469d634c565caed010
I20260709 13:18:32.662374 19406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92874c9d488048f29020dca908fb0427"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
mode: GRACEFUL
new_leader_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:56026
I20260709 13:18:32.662808 19406 raft_consensus.cc:606] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Received request to transfer leadership to TS ca787ea0df384e469d634c565caed010
I20260709 13:18:32.663756 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:32.664206 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:32.664752 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:32.831653 19744 raft_consensus.cc:993] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4: : Instructing follower ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:32.832190 19762 raft_consensus.cc:1081] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Signalling peer ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:32.833663 19480 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92874c9d488048f29020dca908fb0427"
dest_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:18:32.834110 19480 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.834421 19480 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.837496 19744 raft_consensus.cc:993] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4: : Instructing follower ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:32.837970 19762 raft_consensus.cc:1081] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Signalling peer ca787ea0df384e469d634c565caed010 to start an election
I20260709 13:18:32.839421 19481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0"
dest_uuid: "ca787ea0df384e469d634c565caed010"
 from {username='slave'} at 127.0.0.1:46216
I20260709 13:18:32.839852 19481 raft_consensus.cc:493] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.840129 19481 raft_consensus.cc:3060] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.840047 19480 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.841472 19480 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 4 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:32.842257 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:32.842691 19681 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.845341 19481 raft_consensus.cc:515] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.846040 19406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:32.846623 19406 raft_consensus.cc:3055] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:32.846978 19406 raft_consensus.cc:740] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:32.847194 19481 leader_election.cc:290] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673)
I20260709 13:18:32.847888 19406 consensus_queue.cc:260] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.848146 19405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
I20260709 13:18:32.848718 19405 raft_consensus.cc:3055] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:32.849006 19405 raft_consensus.cc:740] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:32.848919 19681 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 4.
I20260709 13:18:32.849750 19680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52e3a1dbf8a54a39a92579d94298f8f0" candidate_uuid: "ca787ea0df384e469d634c565caed010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:32.849670 19405 consensus_queue.cc:260] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.848907 19406 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.850497 19680 raft_consensus.cc:3060] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.850747 19445 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:32.851024 19405 raft_consensus.cc:3060] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:32.851615 19756 raft_consensus.cc:2804] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:32.857473 19680 raft_consensus.cc:2468] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:32.858369 19405 raft_consensus.cc:2468] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 2.
I20260709 13:18:32.859043 19445 leader_election.cc:304] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca787ea0df384e469d634c565caed010, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:32.860770 19776 raft_consensus.cc:2804] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:32.861295 19776 raft_consensus.cc:697] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:32.861209 19756 raft_consensus.cc:697] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 LEADER]: Becoming Leader. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:32.862089 19406 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca787ea0df384e469d634c565caed010 in term 4.
I20260709 13:18:32.862285 19776 consensus_queue.cc:237] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.862634 19756 consensus_queue.cc:237] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:32.869753 19228 catalog_manager.cc:5697] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 3 to 4, leader changed from 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66) to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 4 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.884433 19228 catalog_manager.cc:5697] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 reported cstate change: term changed from 1 to 2, leader changed from 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66) to ca787ea0df384e469d634c565caed010 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "ca787ea0df384e469d634c565caed010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.317235 19406 raft_consensus.cc:1275] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:33.318636 19756 consensus_queue.cc:1048] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.328606 19680 raft_consensus.cc:1275] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 4 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:33.330245 19756 consensus_queue.cc:1048] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.336320 19680 raft_consensus.cc:1275] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:33.338059 19776 consensus_queue.cc:1048] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:33.348337 19406 raft_consensus.cc:1275] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Refusing update from remote peer ca787ea0df384e469d634c565caed010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:33.349563 19756 consensus_queue.cc:1048] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:33.643679 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:33.671269 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:33.680471 19481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92874c9d488048f29020dca908fb0427"
dest_uuid: "ca787ea0df384e469d634c565caed010"
mode: GRACEFUL
new_leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:40510
I20260709 13:18:33.681139 19481 raft_consensus.cc:606] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 LEADER]: Received request to transfer leadership to TS 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:33.683840 19481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff"
dest_uuid: "ca787ea0df384e469d634c565caed010"
mode: GRACEFUL
new_leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:40510
I20260709 13:18:33.684283 19481 raft_consensus.cc:606] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Received request to transfer leadership to TS 93c3a69dd97f44f481ff44b3b667f3b4
I20260709 13:18:33.685320 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:33.685741 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:33.686348 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:33.717525 19775 raft_consensus.cc:993] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010: : Instructing follower 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:33.718149 19756 raft_consensus.cc:1081] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Signalling peer 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:33.720196 19406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:35156
I20260709 13:18:33.720849 19406 raft_consensus.cc:493] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.721335 19406 raft_consensus.cc:3060] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.722922 19756 raft_consensus.cc:993] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: : Instructing follower 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:33.723497 19775 raft_consensus.cc:1081] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 LEADER]: Signalling peer 93c3a69dd97f44f481ff44b3b667f3b4 to start an election
I20260709 13:18:33.725119 19405 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92874c9d488048f29020dca908fb0427"
dest_uuid: "93c3a69dd97f44f481ff44b3b667f3b4"
 from {username='slave'} at 127.0.0.1:35156
I20260709 13:18:33.725539 19405 raft_consensus.cc:493] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.725807 19405 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.730238 19406 raft_consensus.cc:515] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:33.732425 19406 leader_election.cc:290] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 election: Requested vote from peers ca787ea0df384e469d634c565caed010 (127.10.4.67:39861), ef42340151804641b647fd666754cb24 (127.10.4.65:46655)
I20260709 13:18:33.732961 19405 raft_consensus.cc:515] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:33.733472 19481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:33.734337 19481 raft_consensus.cc:3055] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:33.734584 19481 raft_consensus.cc:740] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:33.734910 19405 leader_election.cc:290] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 5 election: Requested vote from peers ef42340151804641b647fd666754cb24 (127.10.4.65:46655), ca787ea0df384e469d634c565caed010 (127.10.4.67:39861)
I20260709 13:18:33.736034 19680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:33.736220 19480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92874c9d488048f29020dca908fb0427" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "ca787ea0df384e469d634c565caed010"
I20260709 13:18:33.733722 19681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d605fd708e0e42ea8c5d9a117b1ddbff" candidate_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef42340151804641b647fd666754cb24"
I20260709 13:18:33.735791 19481 consensus_queue.cc:260] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:33.736792 19680 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.736827 19480 raft_consensus.cc:3055] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:33.737005 19681 raft_consensus.cc:3060] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.737553 19480 raft_consensus.cc:740] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ca787ea0df384e469d634c565caed010, State: Running, Role: LEADER
I20260709 13:18:33.738300 19481 raft_consensus.cc:3060] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:33.738785 19480 consensus_queue.cc:260] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:33.740449 19480 raft_consensus.cc:3060] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.747591 19681 raft_consensus.cc:2468] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 2.
I20260709 13:18:33.748423 19481 raft_consensus.cc:2468] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 2.
I20260709 13:18:33.747690 19680 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 5.
I20260709 13:18:33.749295 19371 leader_election.cc:304] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:33.750068 19480 raft_consensus.cc:2468] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c3a69dd97f44f481ff44b3b667f3b4 in term 5.
I20260709 13:18:33.750555 19762 raft_consensus.cc:2804] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:33.750797 19371 leader_election.cc:304] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c3a69dd97f44f481ff44b3b667f3b4, ef42340151804641b647fd666754cb24; no voters: 
I20260709 13:18:33.751788 19762 raft_consensus.cc:697] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:33.752600 19792 raft_consensus.cc:2804] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:33.753159 19792 raft_consensus.cc:697] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 5 LEADER]: Becoming Leader. State: Replica: 93c3a69dd97f44f481ff44b3b667f3b4, State: Running, Role: LEADER
I20260709 13:18:33.753055 19762 consensus_queue.cc:237] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } }
I20260709 13:18:33.754455 19792 consensus_queue.cc:237] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } }
I20260709 13:18:33.762091 19229 catalog_manager.cc:5697] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 1 to 2, leader changed from ca787ea0df384e469d634c565caed010 (127.10.4.67) to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.780135 19229 catalog_manager.cc:5697] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 reported cstate change: term changed from 4 to 5, leader changed from ca787ea0df384e469d634c565caed010 (127.10.4.67) to 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66). New cstate: current_term: 5 leader_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c3a69dd97f44f481ff44b3b667f3b4" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 34673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:34.236598 19680 raft_consensus.cc:1275] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:34.237936 19792 consensus_queue.cc:1048] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.249670 19480 raft_consensus.cc:1275] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:34.251312 19762 consensus_queue.cc:1048] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:34.273730 19480 raft_consensus.cc:1275] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 5 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:34.274904 19762 consensus_queue.cc:1048] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca787ea0df384e469d634c565caed010" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:34.284735 19680 raft_consensus.cc:1275] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 5 FOLLOWER]: Refusing update from remote peer 93c3a69dd97f44f481ff44b3b667f3b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:34.286511 19791 consensus_queue.cc:1048] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef42340151804641b647fd666754cb24" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 46655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:18:34.644670 19277 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:18:34.692351 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:34.720297 10257 tablet_server.cc:179] TabletServer@127.10.4.65:46655 shutting down...
W20260709 13:18:34.730358 19371 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 69 similar messages]
W20260709 13:18:34.733615 19371 consensus_peers.cc:597] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.743126 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:34.744174 10257 tablet_replica.cc:333] stopping tablet replica
W20260709 13:18:34.744692 19445 consensus_peers.cc:597] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.745143 10257 raft_consensus.cc:2243] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.745743 10257 raft_consensus.cc:2272] T 52e3a1dbf8a54a39a92579d94298f8f0 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.748422 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.749054 10257 raft_consensus.cc:2243] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.749557 10257 raft_consensus.cc:2272] T 0a20484fdf86439b806bc0b08e492fa8 P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.751825 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.752411 10257 raft_consensus.cc:2243] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.752871 10257 raft_consensus.cc:2272] T 6ed3c39b2d9a4413ad19c07d583a1582 P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.755414 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.755965 10257 raft_consensus.cc:2243] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.756448 10257 raft_consensus.cc:2272] T 587a4db012a54f76a3a1889c1862a1e4 P ef42340151804641b647fd666754cb24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.759317 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.760061 10257 raft_consensus.cc:2243] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.760517 10257 raft_consensus.cc:2272] T c2afe64a363b452fbac62c22914151fd P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.762812 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.763316 10257 raft_consensus.cc:2243] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.763747 10257 raft_consensus.cc:2272] T d605fd708e0e42ea8c5d9a117b1ddbff P ef42340151804641b647fd666754cb24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.766224 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.766733 10257 raft_consensus.cc:2243] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.767205 10257 raft_consensus.cc:2272] T 92874c9d488048f29020dca908fb0427 P ef42340151804641b647fd666754cb24 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.770382 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.770887 10257 raft_consensus.cc:2243] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.771329 10257 raft_consensus.cc:2272] T 46e70c8fee654c7d8a3a28f64d48482e P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.773473 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.773936 10257 raft_consensus.cc:2243] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.774356 10257 raft_consensus.cc:2272] T 3b3f28ae190e415b8e6f445b64db75fa P ef42340151804641b647fd666754cb24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.776515 10257 tablet_replica.cc:333] stopping tablet replica
I20260709 13:18:34.776965 10257 raft_consensus.cc:2243] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.777415 10257 raft_consensus.cc:2272] T 72074f684d42435799b0418ad21fc105 P ef42340151804641b647fd666754cb24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.812393 10257 tablet_server.cc:196] TabletServer@127.10.4.65:46655 shutdown complete.
W20260709 13:18:34.833699 19371 consensus_peers.cc:597] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.842926 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
W20260709 13:18:34.844538 19371 consensus_peers.cc:597] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:34.857596 19371 consensus_peers.cc:597] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 -> Peer ef42340151804641b647fd666754cb24 (127.10.4.65:46655): Couldn't send request to peer ef42340151804641b647fd666754cb24. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:46655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.868932 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:34.869702 10257 tablet_replica.cc:333] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.870455 10257 raft_consensus.cc:2243] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.871462 10257 raft_consensus.cc:2272] T 6ed3c39b2d9a4413ad19c07d583a1582 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.874095 10257 tablet_replica.cc:333] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.874735 10257 raft_consensus.cc:2243] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.875257 10257 raft_consensus.cc:2272] T 0a20484fdf86439b806bc0b08e492fa8 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.877560 10257 tablet_replica.cc:333] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.878130 10257 raft_consensus.cc:2243] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.879026 10257 raft_consensus.cc:2272] T 587a4db012a54f76a3a1889c1862a1e4 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.881294 10257 tablet_replica.cc:333] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.881870 10257 raft_consensus.cc:2243] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.882315 10257 raft_consensus.cc:2272] T 3b3f28ae190e415b8e6f445b64db75fa P 93c3a69dd97f44f481ff44b3b667f3b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.884366 10257 tablet_replica.cc:333] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.884899 10257 raft_consensus.cc:2243] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.885918 10257 raft_consensus.cc:2272] T 46e70c8fee654c7d8a3a28f64d48482e P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.888222 10257 tablet_replica.cc:333] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.888754 10257 raft_consensus.cc:2243] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.889632 10257 raft_consensus.cc:2272] T 72074f684d42435799b0418ad21fc105 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.892156 10257 tablet_replica.cc:333] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.892625 10257 raft_consensus.cc:2243] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.893334 10257 raft_consensus.cc:2272] T d605fd708e0e42ea8c5d9a117b1ddbff P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.896371 10257 tablet_replica.cc:333] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.896929 10257 raft_consensus.cc:2243] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.897356 10257 raft_consensus.cc:2272] T c2afe64a363b452fbac62c22914151fd P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.899372 10257 tablet_replica.cc:333] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.899966 10257 raft_consensus.cc:2243] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.900730 10257 raft_consensus.cc:2272] T 92874c9d488048f29020dca908fb0427 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.903925 10257 tablet_replica.cc:333] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4: stopping tablet replica
I20260709 13:18:34.904407 10257 raft_consensus.cc:2243] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.904856 10257 raft_consensus.cc:2272] T 52e3a1dbf8a54a39a92579d94298f8f0 P 93c3a69dd97f44f481ff44b3b667f3b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.937894 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
I20260709 13:18:34.961676 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
I20260709 13:18:34.988627 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:34.989480 10257 tablet_replica.cc:333] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010: stopping tablet replica
W20260709 13:18:34.989724 19443 consensus_peers.cc:597] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 -> Peer 93c3a69dd97f44f481ff44b3b667f3b4 (127.10.4.66:34673): Couldn't send request to peer 93c3a69dd97f44f481ff44b3b667f3b4. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:34673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:34.990332 10257 raft_consensus.cc:2243] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.991235 10257 raft_consensus.cc:2272] T 52e3a1dbf8a54a39a92579d94298f8f0 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.993456 10257 tablet_replica.cc:333] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:34.994098 10257 raft_consensus.cc:2243] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:34.994629 10257 raft_consensus.cc:2272] T 72074f684d42435799b0418ad21fc105 P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:34.996923 10257 tablet_replica.cc:333] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:34.997465 10257 raft_consensus.cc:2243] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:34.998311 10257 raft_consensus.cc:2272] T 3b3f28ae190e415b8e6f445b64db75fa P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.000608 10257 tablet_replica.cc:333] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:35.001204 10257 raft_consensus.cc:2243] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:35.001644 10257 raft_consensus.cc:2272] T 46e70c8fee654c7d8a3a28f64d48482e P ca787ea0df384e469d634c565caed010 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.003973 10257 tablet_replica.cc:333] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:35.004523 10257 raft_consensus.cc:2243] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:35.005010 10257 raft_consensus.cc:2272] T 587a4db012a54f76a3a1889c1862a1e4 P ca787ea0df384e469d634c565caed010 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.007091 10257 tablet_replica.cc:333] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:35.007596 10257 raft_consensus.cc:2243] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:35.008081 10257 raft_consensus.cc:2272] T 6ed3c39b2d9a4413ad19c07d583a1582 P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.010063 10257 tablet_replica.cc:333] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:35.010574 10257 raft_consensus.cc:2243] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:35.011471 10257 raft_consensus.cc:2272] T 0a20484fdf86439b806bc0b08e492fa8 P ca787ea0df384e469d634c565caed010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.013625 10257 tablet_replica.cc:333] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:35.014079 10257 raft_consensus.cc:2243] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:35.014891 10257 raft_consensus.cc:2272] T c2afe64a363b452fbac62c22914151fd P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.017069 10257 tablet_replica.cc:333] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:35.017520 10257 raft_consensus.cc:2243] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:35.017908 10257 raft_consensus.cc:2272] T 92874c9d488048f29020dca908fb0427 P ca787ea0df384e469d634c565caed010 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.020200 10257 tablet_replica.cc:333] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010: stopping tablet replica
I20260709 13:18:35.020658 10257 raft_consensus.cc:2243] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:35.021014 10257 raft_consensus.cc:2272] T d605fd708e0e42ea8c5d9a117b1ddbff P ca787ea0df384e469d634c565caed010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.051602 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
I20260709 13:18:35.076912 10257 master.cc:562] Master@127.10.4.126:34531 shutting down...
I20260709 13:18:35.103952 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:35.104511 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:35.104830 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6f13b409408b4d53968cc4195eb6302c: stopping tablet replica
I20260709 13:18:35.124235 10257 master.cc:584] Master@127.10.4.126:34531 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12679 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:18:35.166142 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:41137
I20260709 13:18:35.167239 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:35.172708 19811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:35.173660 19812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:35.173827 19814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:35.174481 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:35.175307 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:35.175526 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:35.175648 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603115175636 us; error 0 us; skew 500 ppm
I20260709 13:18:35.176131 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:35.178293 10257 webserver.cc:533] Webserver started at http://127.10.4.126:45603/ using document root <none> and password file <none>
I20260709 13:18:35.178771 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:35.178920 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:35.179133 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:35.180292 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "928a02b05f3844c29c30005038559871"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.184528 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:18:35.187817 19819 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.188524 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:18:35.188813 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "928a02b05f3844c29c30005038559871"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.189055 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:35.199792 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:35.200901 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:35.239038 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:41137
I20260709 13:18:35.239143 19870 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:41137 every 8 connection(s)
I20260709 13:18:35.242865 19871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.253489 19871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871: Bootstrap starting.
I20260709 13:18:35.258167 19871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:35.262496 19871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871: No bootstrap required, opened a new log
I20260709 13:18:35.264694 19871 raft_consensus.cc:359] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928a02b05f3844c29c30005038559871" member_type: VOTER }
I20260709 13:18:35.265061 19871 raft_consensus.cc:385] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:35.265236 19871 raft_consensus.cc:740] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a02b05f3844c29c30005038559871, State: Initialized, Role: FOLLOWER
I20260709 13:18:35.265754 19871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928a02b05f3844c29c30005038559871" member_type: VOTER }
I20260709 13:18:35.266163 19871 raft_consensus.cc:399] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:18:35.266352 19871 raft_consensus.cc:493] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:18:35.266558 19871 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:35.271239 19871 raft_consensus.cc:515] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928a02b05f3844c29c30005038559871" member_type: VOTER }
I20260709 13:18:35.271720 19871 leader_election.cc:304] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 928a02b05f3844c29c30005038559871; no voters: 
I20260709 13:18:35.272708 19871 leader_election.cc:290] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:18:35.273044 19874 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:35.274415 19874 raft_consensus.cc:697] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 1 LEADER]: Becoming Leader. State: Replica: 928a02b05f3844c29c30005038559871, State: Running, Role: LEADER
I20260709 13:18:35.275043 19874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928a02b05f3844c29c30005038559871" member_type: VOTER }
I20260709 13:18:35.275476 19871 sys_catalog.cc:565] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:18:35.279459 19875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "928a02b05f3844c29c30005038559871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928a02b05f3844c29c30005038559871" member_type: VOTER } }
I20260709 13:18:35.279561 19876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 928a02b05f3844c29c30005038559871. Latest consensus state: current_term: 1 leader_uuid: "928a02b05f3844c29c30005038559871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928a02b05f3844c29c30005038559871" member_type: VOTER } }
I20260709 13:18:35.280349 19876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:35.280957 19875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:35.286732 19882 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:18:35.293998 19882 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:18:35.294871 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:18:35.302517 19882 catalog_manager.cc:1383] Generated new cluster ID: ff628d8e8e1d40f98625ec764b3a7d8e
I20260709 13:18:35.302763 19882 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:18:35.322108 19882 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:18:35.323386 19882 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:18:35.335841 19882 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871: Generated new TSK 0
I20260709 13:18:35.336388 19882 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:18:35.361742 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:35.367717 19892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:35.368685 19893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:35.369915 19895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:35.370195 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:35.371209 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:35.371387 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:35.371554 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603115371540 us; error 0 us; skew 500 ppm
I20260709 13:18:35.372023 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:35.374092 10257 webserver.cc:533] Webserver started at http://127.10.4.65:38603/ using document root <none> and password file <none>
I20260709 13:18:35.374498 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:35.374657 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:35.374857 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:35.375906 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "33761e1ed99540f88d2a3e75fa3b6756"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.380195 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:35.383258 19900 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.384009 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:18:35.384258 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "33761e1ed99540f88d2a3e75fa3b6756"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.384512 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:35.397336 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:35.398510 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:35.399923 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:35.402313 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:35.402496 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.402680 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:35.402824 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.441522 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:45391
I20260709 13:18:35.441609 19962 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:45391 every 8 connection(s)
I20260709 13:18:35.446521 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:35.453444 19967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:35.454842 19968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:35.459444 19970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:35.459744 19963 heartbeater.cc:344] Connected to a master server at 127.10.4.126:41137
I20260709 13:18:35.460388 19963 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:35.461477 19963 heartbeater.cc:507] Master 127.10.4.126:41137 requested a full tablet report, sending...
I20260709 13:18:35.463666 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:35.464039 19836 ts_manager.cc:194] Registered new tserver with Master: 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65:45391)
I20260709 13:18:35.464552 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:35.464809 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:35.464960 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603115464944 us; error 0 us; skew 500 ppm
I20260709 13:18:35.465529 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:35.466013 19836 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50604
I20260709 13:18:35.468114 10257 webserver.cc:533] Webserver started at http://127.10.4.66:35103/ using document root <none> and password file <none>
I20260709 13:18:35.468806 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:35.469017 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:35.469225 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:35.470288 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.474498 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:18:35.477823 19975 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.478526 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:18:35.478790 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.479041 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:35.502365 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:35.503573 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:35.505191 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:35.507393 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:35.507633 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.507853 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:35.507998 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.547753 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:45061
I20260709 13:18:35.547852 20037 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:45061 every 8 connection(s)
I20260709 13:18:35.552613 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:35.560258 20041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:35.560915 20042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:35.564496 20038 heartbeater.cc:344] Connected to a master server at 127.10.4.126:41137
I20260709 13:18:35.565028 20038 heartbeater.cc:461] Registering TS with master...
W20260709 13:18:35.565655 20044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:35.565948 20038 heartbeater.cc:507] Master 127.10.4.126:41137 requested a full tablet report, sending...
I20260709 13:18:35.566035 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:35.567178 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:35.567384 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:35.567597 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603115567582 us; error 0 us; skew 500 ppm
I20260709 13:18:35.568259 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:35.568526 19836 ts_manager.cc:194] Registered new tserver with Master: 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061)
I20260709 13:18:35.570195 19836 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50618
I20260709 13:18:35.571319 10257 webserver.cc:533] Webserver started at http://127.10.4.67:42681/ using document root <none> and password file <none>
I20260709 13:18:35.572062 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:35.572319 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:35.572624 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:35.573797 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "463c3b1e2fbb4b69af07fda5335853c6"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.578302 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 13:18:35.581671 20049 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.582417 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:18:35.582700 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "463c3b1e2fbb4b69af07fda5335853c6"
format_stamp: "Formatted at 2026-07-09 13:18:35 on dist-test-slave-gkm9"
I20260709 13:18:35.582959 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:35.632750 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:35.633961 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:35.635466 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:35.637687 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:35.637882 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.638111 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:35.638262 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:35.677466 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:40127
I20260709 13:18:35.677580 20111 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:40127 every 8 connection(s)
I20260709 13:18:35.691150 20112 heartbeater.cc:344] Connected to a master server at 127.10.4.126:41137
I20260709 13:18:35.691548 20112 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:35.692256 20112 heartbeater.cc:507] Master 127.10.4.126:41137 requested a full tablet report, sending...
I20260709 13:18:35.694132 19836 ts_manager.cc:194] Registered new tserver with Master: 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:35.695077 10257 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014379719s
I20260709 13:18:35.695556 19836 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50620
I20260709 13:18:35.720458 19836 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50626:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:35.722667 19836 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:35.770963 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 6834d4df28a04d7892f4510fb1903822 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=42b9c80a69104429b93d6d02b6b5c24a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:18:35.771124 20002 tablet_service.cc:1511] Processing CreateTablet for tablet 772f765b5d4d43ce91bae805131a5725 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=42b9c80a69104429b93d6d02b6b5c24a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:18:35.772778 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6834d4df28a04d7892f4510fb1903822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.773352 20002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772f765b5d4d43ce91bae805131a5725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.780509 20077 tablet_service.cc:1511] Processing CreateTablet for tablet 6834d4df28a04d7892f4510fb1903822 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=42b9c80a69104429b93d6d02b6b5c24a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:18:35.782105 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6834d4df28a04d7892f4510fb1903822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.785645 20076 tablet_service.cc:1511] Processing CreateTablet for tablet 772f765b5d4d43ce91bae805131a5725 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=42b9c80a69104429b93d6d02b6b5c24a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:18:35.787936 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772f765b5d4d43ce91bae805131a5725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.793910 19928 tablet_service.cc:1511] Processing CreateTablet for tablet 6834d4df28a04d7892f4510fb1903822 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=42b9c80a69104429b93d6d02b6b5c24a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:18:35.796551 19928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6834d4df28a04d7892f4510fb1903822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.799083 19927 tablet_service.cc:1511] Processing CreateTablet for tablet 772f765b5d4d43ce91bae805131a5725 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=42b9c80a69104429b93d6d02b6b5c24a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:18:35.800369 19927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772f765b5d4d43ce91bae805131a5725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:35.805074 20127 tablet_bootstrap.cc:492] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c: Bootstrap starting.
I20260709 13:18:35.813722 20127 tablet_bootstrap.cc:654] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:35.814858 20128 tablet_bootstrap.cc:492] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6: Bootstrap starting.
I20260709 13:18:35.821306 20130 tablet_bootstrap.cc:492] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: Bootstrap starting.
I20260709 13:18:35.824728 20128 tablet_bootstrap.cc:654] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:35.827072 20130 tablet_bootstrap.cc:654] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:35.829051 20127 tablet_bootstrap.cc:492] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c: No bootstrap required, opened a new log
I20260709 13:18:35.829598 20128 tablet_bootstrap.cc:492] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6: No bootstrap required, opened a new log
I20260709 13:18:35.829617 20127 ts_tablet_manager.cc:1403] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.000s
I20260709 13:18:35.830339 20128 ts_tablet_manager.cc:1403] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:18:35.831811 20130 tablet_bootstrap.cc:492] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: No bootstrap required, opened a new log
I20260709 13:18:35.832273 20130 ts_tablet_manager.cc:1403] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: Time spent bootstrapping tablet: real 0.011s	user 0.004s	sys 0.006s
I20260709 13:18:35.833212 20127 raft_consensus.cc:359] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.833366 20128 raft_consensus.cc:359] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.834143 20127 raft_consensus.cc:385] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:35.834290 20128 raft_consensus.cc:385] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:35.834539 20127 raft_consensus.cc:740] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b9b02685874c85b6bc1b25ccc45b1c, State: Initialized, Role: FOLLOWER
I20260709 13:18:35.834625 20128 raft_consensus.cc:740] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463c3b1e2fbb4b69af07fda5335853c6, State: Initialized, Role: FOLLOWER
I20260709 13:18:35.834632 20130 raft_consensus.cc:359] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.835561 20130 raft_consensus.cc:385] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:35.835879 20130 raft_consensus.cc:740] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Initialized, Role: FOLLOWER
I20260709 13:18:35.835644 20128 consensus_queue.cc:260] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.835597 20127 consensus_queue.cc:260] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.836663 20130 consensus_queue.cc:260] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.839031 20128 ts_tablet_manager.cc:1434] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:18:35.839295 20038 heartbeater.cc:499] Master 127.10.4.126:41137 was elected leader, sending a full tablet report...
I20260709 13:18:35.840288 20128 tablet_bootstrap.cc:492] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6: Bootstrap starting.
I20260709 13:18:35.839358 20112 heartbeater.cc:499] Master 127.10.4.126:41137 was elected leader, sending a full tablet report...
I20260709 13:18:35.847841 20128 tablet_bootstrap.cc:654] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:35.848500 19963 heartbeater.cc:499] Master 127.10.4.126:41137 was elected leader, sending a full tablet report...
I20260709 13:18:35.848917 20130 ts_tablet_manager.cc:1434] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: Time spent starting tablet: real 0.016s	user 0.000s	sys 0.012s
I20260709 13:18:35.849781 20130 tablet_bootstrap.cc:492] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756: Bootstrap starting.
I20260709 13:18:35.854923 20130 tablet_bootstrap.cc:654] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:35.855295 20128 tablet_bootstrap.cc:492] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6: No bootstrap required, opened a new log
I20260709 13:18:35.855867 20128 ts_tablet_manager.cc:1403] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.008s
I20260709 13:18:35.858933 20128 raft_consensus.cc:359] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.859720 20128 raft_consensus.cc:385] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:35.860026 20128 raft_consensus.cc:740] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463c3b1e2fbb4b69af07fda5335853c6, State: Initialized, Role: FOLLOWER
I20260709 13:18:35.861048 20134 raft_consensus.cc:493] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.860823 20128 consensus_queue.cc:260] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.861569 20134 raft_consensus.cc:515] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.863035 20128 ts_tablet_manager.cc:1434] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:35.867025 20134 leader_election.cc:290] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65:45391), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:35.875813 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822" candidate_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33761e1ed99540f88d2a3e75fa3b6756" is_pre_election: true
I20260709 13:18:35.876771 19938 raft_consensus.cc:2468] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32b9b02685874c85b6bc1b25ccc45b1c in term 0.
I20260709 13:18:35.877508 20130 tablet_bootstrap.cc:492] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756: No bootstrap required, opened a new log
I20260709 13:18:35.878262 19977 leader_election.cc:304] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32b9b02685874c85b6bc1b25ccc45b1c, 33761e1ed99540f88d2a3e75fa3b6756; no voters: 
I20260709 13:18:35.879057 20130 ts_tablet_manager.cc:1403] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.013s
I20260709 13:18:35.879365 20134 raft_consensus.cc:2804] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:35.879860 20134 raft_consensus.cc:493] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.880272 20134 raft_consensus.cc:3060] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:35.846700 20127 ts_tablet_manager.cc:1434] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:18:35.882337 20130 raft_consensus.cc:359] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.883181 20130 raft_consensus.cc:385] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:35.883577 20130 raft_consensus.cc:740] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Initialized, Role: FOLLOWER
I20260709 13:18:35.883432 20127 tablet_bootstrap.cc:492] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c: Bootstrap starting.
I20260709 13:18:35.884526 20130 consensus_queue.cc:260] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.886801 20130 ts_tablet_manager.cc:1434] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:18:35.890448 20134 raft_consensus.cc:515] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.894681 20134 leader_election.cc:290] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [CANDIDATE]: Term 1 election: Requested vote from peers 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65:45391), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:35.895612 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822" candidate_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33761e1ed99540f88d2a3e75fa3b6756"
I20260709 13:18:35.896278 19938 raft_consensus.cc:3060] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:35.897507 20127 tablet_bootstrap.cc:654] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:35.899716 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822" candidate_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6" is_pre_election: true
I20260709 13:18:35.900467 20087 raft_consensus.cc:2468] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32b9b02685874c85b6bc1b25ccc45b1c in term 0.
I20260709 13:18:35.901680 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822" candidate_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
I20260709 13:18:35.902349 20086 raft_consensus.cc:3060] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:35.905212 19938 raft_consensus.cc:2468] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32b9b02685874c85b6bc1b25ccc45b1c in term 1.
I20260709 13:18:35.906517 19977 leader_election.cc:304] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32b9b02685874c85b6bc1b25ccc45b1c, 33761e1ed99540f88d2a3e75fa3b6756; no voters: 
I20260709 13:18:35.907461 20134 raft_consensus.cc:2804] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:35.909600 20134 raft_consensus.cc:697] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 LEADER]: Becoming Leader. State: Replica: 32b9b02685874c85b6bc1b25ccc45b1c, State: Running, Role: LEADER
I20260709 13:18:35.910570 20086 raft_consensus.cc:2468] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32b9b02685874c85b6bc1b25ccc45b1c in term 1.
I20260709 13:18:35.910480 20134 consensus_queue.cc:237] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.911166 20127 tablet_bootstrap.cc:492] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c: No bootstrap required, opened a new log
I20260709 13:18:35.911803 20127 ts_tablet_manager.cc:1403] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.014s
I20260709 13:18:35.914377 20127 raft_consensus.cc:359] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.915179 20127 raft_consensus.cc:385] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:35.915513 20127 raft_consensus.cc:740] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b9b02685874c85b6bc1b25ccc45b1c, State: Initialized, Role: FOLLOWER
I20260709 13:18:35.916194 20127 consensus_queue.cc:260] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.918066 20127 ts_tablet_manager.cc:1434] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:18:35.918766 19835 catalog_manager.cc:5697] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66). New cstate: current_term: 1 leader_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:35.996841 20135 raft_consensus.cc:493] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.997380 20135 raft_consensus.cc:515] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:35.999310 20135 leader_election.cc:290] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.009709 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6" is_pre_election: true
I20260709 13:18:36.009720 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" is_pre_election: true
I20260709 13:18:36.010649 20086 raft_consensus.cc:2468] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 0.
I20260709 13:18:36.010731 20013 raft_consensus.cc:2468] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 0.
I20260709 13:18:36.012027 19901 leader_election.cc:304] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33761e1ed99540f88d2a3e75fa3b6756, 463c3b1e2fbb4b69af07fda5335853c6; no voters: 
I20260709 13:18:36.012926 20135 raft_consensus.cc:2804] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:36.013218 20135 raft_consensus.cc:493] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.013464 20135 raft_consensus.cc:3060] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.018563 20135 raft_consensus.cc:515] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.020120 20135 leader_election.cc:290] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 election: Requested vote from peers 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.020911 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
I20260709 13:18:36.021059 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
I20260709 13:18:36.021417 20013 raft_consensus.cc:3060] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.021590 20086 raft_consensus.cc:3060] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.026437 20086 raft_consensus.cc:2468] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 1.
I20260709 13:18:36.026705 20013 raft_consensus.cc:2468] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 1.
I20260709 13:18:36.027443 19901 leader_election.cc:304] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33761e1ed99540f88d2a3e75fa3b6756, 463c3b1e2fbb4b69af07fda5335853c6; no voters: 
I20260709 13:18:36.028286 20135 raft_consensus.cc:2804] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:36.029366 20135 raft_consensus.cc:697] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Becoming Leader. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Running, Role: LEADER
I20260709 13:18:36.030138 20135 consensus_queue.cc:237] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.036626 19836 catalog_manager.cc:5697] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "33761e1ed99540f88d2a3e75fa3b6756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.093709 19836 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50636:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:36.095597 19836 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:36.116454 19928 tablet_service.cc:1511] Processing CreateTablet for tablet 47e7a8d209014afc8205521c6ca3392f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=286550e5e31845f9af4452fc4a9942b7]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:18:36.117739 19928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e7a8d209014afc8205521c6ca3392f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:36.117952 20003 tablet_service.cc:1511] Processing CreateTablet for tablet 47e7a8d209014afc8205521c6ca3392f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=286550e5e31845f9af4452fc4a9942b7]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:18:36.119218 20003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e7a8d209014afc8205521c6ca3392f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:36.120435 19927 tablet_service.cc:1511] Processing CreateTablet for tablet 08d9237d2f2a47858badc3c4ae5ad260 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=286550e5e31845f9af4452fc4a9942b7]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:18:36.121713 19927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08d9237d2f2a47858badc3c4ae5ad260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:36.123378 20002 tablet_service.cc:1511] Processing CreateTablet for tablet 08d9237d2f2a47858badc3c4ae5ad260 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=286550e5e31845f9af4452fc4a9942b7]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:18:36.124163 20077 tablet_service.cc:1511] Processing CreateTablet for tablet 47e7a8d209014afc8205521c6ca3392f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=286550e5e31845f9af4452fc4a9942b7]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:18:36.124732 20002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08d9237d2f2a47858badc3c4ae5ad260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:36.125414 20077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e7a8d209014afc8205521c6ca3392f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:36.128423 20076 tablet_service.cc:1511] Processing CreateTablet for tablet 08d9237d2f2a47858badc3c4ae5ad260 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=286550e5e31845f9af4452fc4a9942b7]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:18:36.129618 20076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08d9237d2f2a47858badc3c4ae5ad260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:36.134893 20130 tablet_bootstrap.cc:492] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756: Bootstrap starting.
I20260709 13:18:36.138767 20127 tablet_bootstrap.cc:492] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c: Bootstrap starting.
I20260709 13:18:36.141325 20130 tablet_bootstrap.cc:654] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:36.151476 20128 tablet_bootstrap.cc:492] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: Bootstrap starting.
I20260709 13:18:36.152819 20130 tablet_bootstrap.cc:492] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756: No bootstrap required, opened a new log
I20260709 13:18:36.153132 20127 tablet_bootstrap.cc:654] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:36.153353 20130 ts_tablet_manager.cc:1403] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 13:18:36.156440 20130 raft_consensus.cc:359] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.157096 20130 raft_consensus.cc:385] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:36.157379 20130 raft_consensus.cc:740] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Initialized, Role: FOLLOWER
I20260709 13:18:36.158376 20128 tablet_bootstrap.cc:654] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:36.158169 20130 consensus_queue.cc:260] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.162971 20130 ts_tablet_manager.cc:1434] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:18:36.164088 20130 tablet_bootstrap.cc:492] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756: Bootstrap starting.
I20260709 13:18:36.169432 20127 tablet_bootstrap.cc:492] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c: No bootstrap required, opened a new log
I20260709 13:18:36.170089 20127 ts_tablet_manager.cc:1403] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.008s
I20260709 13:18:36.170667 20130 tablet_bootstrap.cc:654] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:36.170679 20128 tablet_bootstrap.cc:492] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: No bootstrap required, opened a new log
I20260709 13:18:36.171679 20128 ts_tablet_manager.cc:1403] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.006s
I20260709 13:18:36.173251 20127 raft_consensus.cc:359] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.174054 20127 raft_consensus.cc:385] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:36.174417 20127 raft_consensus.cc:740] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b9b02685874c85b6bc1b25ccc45b1c, State: Initialized, Role: FOLLOWER
I20260709 13:18:36.174953 20128 raft_consensus.cc:359] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.175825 20128 raft_consensus.cc:385] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:36.175452 20127 consensus_queue.cc:260] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.176138 20128 raft_consensus.cc:740] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463c3b1e2fbb4b69af07fda5335853c6, State: Initialized, Role: FOLLOWER
I20260709 13:18:36.177208 20128 consensus_queue.cc:260] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.177891 20127 ts_tablet_manager.cc:1434] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:36.178632 20130 tablet_bootstrap.cc:492] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756: No bootstrap required, opened a new log
I20260709 13:18:36.179220 20130 ts_tablet_manager.cc:1403] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.009s
I20260709 13:18:36.179483 20128 ts_tablet_manager.cc:1434] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:36.180656 20128 tablet_bootstrap.cc:492] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6: Bootstrap starting.
I20260709 13:18:36.182639 20130 raft_consensus.cc:359] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.183534 20130 raft_consensus.cc:385] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:36.183858 20130 raft_consensus.cc:740] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Initialized, Role: FOLLOWER
I20260709 13:18:36.184759 20130 consensus_queue.cc:260] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.188194 20128 tablet_bootstrap.cc:654] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:36.190697 20127 tablet_bootstrap.cc:492] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c: Bootstrap starting.
I20260709 13:18:36.192451 20130 ts_tablet_manager.cc:1434] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:18:36.198094 20127 tablet_bootstrap.cc:654] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:36.202021 20128 tablet_bootstrap.cc:492] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6: No bootstrap required, opened a new log
I20260709 13:18:36.202553 20128 ts_tablet_manager.cc:1403] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.006s
I20260709 13:18:36.204177 20127 tablet_bootstrap.cc:492] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c: No bootstrap required, opened a new log
I20260709 13:18:36.204619 20127 ts_tablet_manager.cc:1403] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:18:36.205068 20128 raft_consensus.cc:359] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.205638 20128 raft_consensus.cc:385] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:36.205853 20128 raft_consensus.cc:740] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463c3b1e2fbb4b69af07fda5335853c6, State: Initialized, Role: FOLLOWER
I20260709 13:18:36.206434 20128 consensus_queue.cc:260] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.206848 20127 raft_consensus.cc:359] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.207577 20127 raft_consensus.cc:385] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:36.207859 20127 raft_consensus.cc:740] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b9b02685874c85b6bc1b25ccc45b1c, State: Initialized, Role: FOLLOWER
I20260709 13:18:36.208359 20128 ts_tablet_manager.cc:1434] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:18:36.208612 20127 consensus_queue.cc:260] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.210634 20127 ts_tablet_manager.cc:1434] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
W20260709 13:18:36.289075 19884 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:36.377827 20134 consensus_queue.cc:1048] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:36.390055 20160 consensus_queue.cc:1048] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:36.396906 20135 raft_consensus.cc:493] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.397532 20135 raft_consensus.cc:515] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.400408 20135 leader_election.cc:290] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.400897 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6" is_pre_election: true
I20260709 13:18:36.401286 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" is_pre_election: true
I20260709 13:18:36.401631 20086 raft_consensus.cc:2468] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 0.
I20260709 13:18:36.401993 20013 raft_consensus.cc:2468] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 0.
I20260709 13:18:36.402688 19901 leader_election.cc:304] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33761e1ed99540f88d2a3e75fa3b6756, 463c3b1e2fbb4b69af07fda5335853c6; no voters: 
I20260709 13:18:36.403522 20135 raft_consensus.cc:2804] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:36.403807 20135 raft_consensus.cc:493] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.404054 20135 raft_consensus.cc:3060] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.406728 20144 raft_consensus.cc:493] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.407210 20144 raft_consensus.cc:515] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.409350 20144 leader_election.cc:290] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.409914 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e7a8d209014afc8205521c6ca3392f" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" is_pre_election: true
I20260709 13:18:36.410477 20013 raft_consensus.cc:2468] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 0.
I20260709 13:18:36.411005 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e7a8d209014afc8205521c6ca3392f" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6" is_pre_election: true
I20260709 13:18:36.411545 20086 raft_consensus.cc:2468] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 0.
I20260709 13:18:36.411744 19904 leader_election.cc:304] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32b9b02685874c85b6bc1b25ccc45b1c, 33761e1ed99540f88d2a3e75fa3b6756; no voters: 
I20260709 13:18:36.412451 20144 raft_consensus.cc:2804] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:36.412781 20144 raft_consensus.cc:493] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.413051 20144 raft_consensus.cc:3060] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.412812 20135 raft_consensus.cc:515] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.414520 20135 leader_election.cc:290] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 election: Requested vote from peers 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.415299 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
I20260709 13:18:36.415581 20086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
I20260709 13:18:36.415833 20013 raft_consensus.cc:3060] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.416209 20086 raft_consensus.cc:3060] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.419859 20144 raft_consensus.cc:515] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.420630 20013 raft_consensus.cc:2468] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 1.
I20260709 13:18:36.421626 19904 leader_election.cc:304] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32b9b02685874c85b6bc1b25ccc45b1c, 33761e1ed99540f88d2a3e75fa3b6756; no voters: 
I20260709 13:18:36.422044 20144 leader_election.cc:290] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 election: Requested vote from peers 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.422669 20144 raft_consensus.cc:2804] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:36.423048 20086 raft_consensus.cc:2468] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 1.
I20260709 13:18:36.423111 20144 raft_consensus.cc:697] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Becoming Leader. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Running, Role: LEADER
I20260709 13:18:36.423192 20013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e7a8d209014afc8205521c6ca3392f" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
I20260709 13:18:36.423192 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e7a8d209014afc8205521c6ca3392f" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
I20260709 13:18:36.424197 20013 raft_consensus.cc:3060] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.424636 20087 raft_consensus.cc:3060] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:36.424324 20144 consensus_queue.cc:237] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.431116 20013 raft_consensus.cc:2468] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 1.
I20260709 13:18:36.431627 20087 raft_consensus.cc:2468] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 1.
I20260709 13:18:36.432421 19904 leader_election.cc:304] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32b9b02685874c85b6bc1b25ccc45b1c, 33761e1ed99540f88d2a3e75fa3b6756; no voters: 
I20260709 13:18:36.433205 20135 raft_consensus.cc:2804] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:36.433660 20135 raft_consensus.cc:697] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Becoming Leader. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Running, Role: LEADER
I20260709 13:18:36.434420 20135 consensus_queue.cc:237] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.436136 19833 catalog_manager.cc:5697] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "33761e1ed99540f88d2a3e75fa3b6756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.448536 19833 catalog_manager.cc:5697] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "33761e1ed99540f88d2a3e75fa3b6756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.471549 20013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822"
dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
mode: GRACEFUL
new_leader_uuid: "33761e1ed99540f88d2a3e75fa3b6756"
 from {username='slave'} at 127.0.0.1:46384
I20260709 13:18:36.472072 20013 raft_consensus.cc:606] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 LEADER]: Received request to transfer leadership to TS 33761e1ed99540f88d2a3e75fa3b6756
I20260709 13:18:36.484627 19938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725"
dest_uuid: "33761e1ed99540f88d2a3e75fa3b6756"
mode: GRACEFUL
new_leader_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
 from {username='slave'} at 127.0.0.1:56980
I20260709 13:18:36.485163 19938 raft_consensus.cc:606] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Received request to transfer leadership to TS 32b9b02685874c85b6bc1b25ccc45b1c
I20260709 13:18:36.485975 20135 consensus_queue.cc:1048] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:36.492812 20144 raft_consensus.cc:993] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756: : Instructing follower 32b9b02685874c85b6bc1b25ccc45b1c to start an election
I20260709 13:18:36.493502 20135 raft_consensus.cc:1081] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Signalling peer 32b9b02685874c85b6bc1b25ccc45b1c to start an election
I20260709 13:18:36.495888 20012 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725"
dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
 from {username='slave'} at 127.0.0.1:46370
I20260709 13:18:36.496474 20012 raft_consensus.cc:493] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:36.496836 20012 raft_consensus.cc:3060] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.503824 20012 raft_consensus.cc:515] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.504932 20013 raft_consensus.cc:1240] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Rejecting Update request from peer 33761e1ed99540f88d2a3e75fa3b6756 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:36.506696 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725" candidate_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33761e1ed99540f88d2a3e75fa3b6756"
I20260709 13:18:36.507336 19938 raft_consensus.cc:3055] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:36.507673 19938 raft_consensus.cc:740] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Running, Role: LEADER
I20260709 13:18:36.508490 19938 consensus_queue.cc:260] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.509363 20144 consensus_queue.cc:1048] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:36.510394 19938 raft_consensus.cc:3060] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.511830 20012 leader_election.cc:290] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [CANDIDATE]: Term 2 election: Requested vote from peers 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65:45391), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.512856 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772f765b5d4d43ce91bae805131a5725" candidate_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
I20260709 13:18:36.513454 20087 raft_consensus.cc:3060] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.516939 10257 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:18:36.517107 19938 raft_consensus.cc:2468] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32b9b02685874c85b6bc1b25ccc45b1c in term 2.
I20260709 13:18:36.518589 10257 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:18:36.519042 10257 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:18:36.520274 20087 raft_consensus.cc:2468] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32b9b02685874c85b6bc1b25ccc45b1c in term 2.
I20260709 13:18:36.520212 19977 leader_election.cc:304] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32b9b02685874c85b6bc1b25ccc45b1c, 33761e1ed99540f88d2a3e75fa3b6756; no voters: 
I20260709 13:18:36.521345 20160 raft_consensus.cc:2804] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:36.521821 20160 raft_consensus.cc:697] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 LEADER]: Becoming Leader. State: Replica: 32b9b02685874c85b6bc1b25ccc45b1c, State: Running, Role: LEADER
I20260709 13:18:36.522451 20160 consensus_queue.cc:237] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.529148 19836 catalog_manager.cc:5697] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c reported cstate change: term changed from 1 to 2, leader changed from 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65) to 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66). New cstate: current_term: 2 leader_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.535029 19938 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260"
dest_uuid: "33761e1ed99540f88d2a3e75fa3b6756"
mode: GRACEFUL
new_leader_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
 from {username='slave'} at 127.0.0.1:56982
I20260709 13:18:36.535638 19938 raft_consensus.cc:606] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Received request to transfer leadership to TS 463c3b1e2fbb4b69af07fda5335853c6
I20260709 13:18:36.537576 10257 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:18:36.792616 20134 raft_consensus.cc:993] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c: : Instructing follower 33761e1ed99540f88d2a3e75fa3b6756 to start an election
I20260709 13:18:36.793061 20160 raft_consensus.cc:1081] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 LEADER]: Signalling peer 33761e1ed99540f88d2a3e75fa3b6756 to start an election
I20260709 13:18:36.794449 19938 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822"
dest_uuid: "33761e1ed99540f88d2a3e75fa3b6756"
 from {username='slave'} at 127.0.0.1:56968
I20260709 13:18:36.794910 19938 raft_consensus.cc:493] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:36.795177 19938 raft_consensus.cc:3060] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.799862 19938 raft_consensus.cc:515] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.801321 19938 leader_election.cc:290] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 2 election: Requested vote from peers 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061), 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67:40127)
I20260709 13:18:36.802253 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
I20260709 13:18:36.802439 20087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6834d4df28a04d7892f4510fb1903822" candidate_uuid: "33761e1ed99540f88d2a3e75fa3b6756" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
I20260709 13:18:36.802815 20012 raft_consensus.cc:3055] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:36.803035 20087 raft_consensus.cc:3060] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.803128 20012 raft_consensus.cc:740] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 32b9b02685874c85b6bc1b25ccc45b1c, State: Running, Role: LEADER
I20260709 13:18:36.803951 20012 consensus_queue.cc:260] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.804880 20012 raft_consensus.cc:3060] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.807840 20087 raft_consensus.cc:2468] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 2.
I20260709 13:18:36.808853 19901 leader_election.cc:304] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33761e1ed99540f88d2a3e75fa3b6756, 463c3b1e2fbb4b69af07fda5335853c6; no voters: 
I20260709 13:18:36.809516 20144 raft_consensus.cc:2804] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:36.809974 20144 raft_consensus.cc:697] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 LEADER]: Becoming Leader. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Running, Role: LEADER
I20260709 13:18:36.810654 20012 raft_consensus.cc:2468] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33761e1ed99540f88d2a3e75fa3b6756 in term 2.
I20260709 13:18:36.810771 20144 consensus_queue.cc:237] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.817468 19836 catalog_manager.cc:5697] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 reported cstate change: term changed from 1 to 2, leader changed from 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66) to 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "33761e1ed99540f88d2a3e75fa3b6756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.828997 20144 consensus_queue.cc:1048] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:36.839085 20144 consensus_queue.cc:1048] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:36.942087 20144 consensus_queue.cc:1048] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:36.948096 20168 raft_consensus.cc:993] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756: : Instructing follower 463c3b1e2fbb4b69af07fda5335853c6 to start an election
I20260709 13:18:36.948589 20135 raft_consensus.cc:1081] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Signalling peer 463c3b1e2fbb4b69af07fda5335853c6 to start an election
I20260709 13:18:36.950232 20086 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260"
dest_uuid: "463c3b1e2fbb4b69af07fda5335853c6"
 from {username='slave'} at 127.0.0.1:41256
I20260709 13:18:36.950809 20086 raft_consensus.cc:493] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:36.951166 20086 raft_consensus.cc:3060] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.954213 20135 consensus_queue.cc:1048] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:36.958103 20086 raft_consensus.cc:515] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.962301 20086 leader_election.cc:290] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [CANDIDATE]: Term 2 election: Requested vote from peers 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65:45391), 32b9b02685874c85b6bc1b25ccc45b1c (127.10.4.66:45061)
I20260709 13:18:36.973361 19938 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260" candidate_uuid: "463c3b1e2fbb4b69af07fda5335853c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33761e1ed99540f88d2a3e75fa3b6756"
I20260709 13:18:36.973894 19938 raft_consensus.cc:3055] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:36.974241 19938 raft_consensus.cc:740] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33761e1ed99540f88d2a3e75fa3b6756, State: Running, Role: LEADER
I20260709 13:18:36.975049 19938 consensus_queue.cc:260] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.976250 19938 raft_consensus.cc:3060] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.976478 20012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08d9237d2f2a47858badc3c4ae5ad260" candidate_uuid: "463c3b1e2fbb4b69af07fda5335853c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "32b9b02685874c85b6bc1b25ccc45b1c"
I20260709 13:18:36.977056 20012 raft_consensus.cc:3060] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:36.981091 19938 raft_consensus.cc:2468] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 463c3b1e2fbb4b69af07fda5335853c6 in term 2.
I20260709 13:18:36.981736 20012 raft_consensus.cc:2468] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 463c3b1e2fbb4b69af07fda5335853c6 in term 2.
I20260709 13:18:36.982223 20051 leader_election.cc:304] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33761e1ed99540f88d2a3e75fa3b6756, 463c3b1e2fbb4b69af07fda5335853c6; no voters: 
I20260709 13:18:36.982962 20133 raft_consensus.cc:2804] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:36.984011 20133 raft_consensus.cc:697] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 LEADER]: Becoming Leader. State: Replica: 463c3b1e2fbb4b69af07fda5335853c6, State: Running, Role: LEADER
I20260709 13:18:36.984771 20133 consensus_queue.cc:237] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } }
I20260709 13:18:36.990653 19833 catalog_manager.cc:5697] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 reported cstate change: term changed from 1 to 2, leader changed from 33761e1ed99540f88d2a3e75fa3b6756 (127.10.4.65) to 463c3b1e2fbb4b69af07fda5335853c6 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "463c3b1e2fbb4b69af07fda5335853c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32b9b02685874c85b6bc1b25ccc45b1c" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 45061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:37.030920 20086 raft_consensus.cc:1275] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Refusing update from remote peer 32b9b02685874c85b6bc1b25ccc45b1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:37.032155 20160 consensus_queue.cc:1048] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "463c3b1e2fbb4b69af07fda5335853c6" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 40127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:37.043798 19938 raft_consensus.cc:1275] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Refusing update from remote peer 32b9b02685874c85b6bc1b25ccc45b1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:37.045478 20160 consensus_queue.cc:1048] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "33761e1ed99540f88d2a3e75fa3b6756" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 45391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:37.149355 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
I20260709 13:18:37.166206 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:37.166756 10257 tablet_replica.cc:333] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756: stopping tablet replica
I20260709 13:18:37.167264 10257 raft_consensus.cc:2243] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.167719 10257 raft_consensus.cc:2272] T 08d9237d2f2a47858badc3c4ae5ad260 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.169746 10257 tablet_replica.cc:333] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756: stopping tablet replica
I20260709 13:18:37.170236 10257 raft_consensus.cc:2243] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:37.170972 10257 raft_consensus.cc:2272] T 47e7a8d209014afc8205521c6ca3392f P 33761e1ed99540f88d2a3e75fa3b6756 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.172995 10257 tablet_replica.cc:333] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756: stopping tablet replica
I20260709 13:18:37.173473 10257 raft_consensus.cc:2243] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.173892 10257 raft_consensus.cc:2272] T 772f765b5d4d43ce91bae805131a5725 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.175959 10257 tablet_replica.cc:333] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: stopping tablet replica
I20260709 13:18:37.176421 10257 raft_consensus.cc:2243] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:37.176967 10257 pending_rounds.cc:70] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: Trying to abort 1 pending ops.
I20260709 13:18:37.177193 10257 pending_rounds.cc:77] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305638366465777664 op_type: NO_OP noop_request { }
I20260709 13:18:37.177446 10257 raft_consensus.cc:2889] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:37.177707 10257 raft_consensus.cc:2272] T 6834d4df28a04d7892f4510fb1903822 P 33761e1ed99540f88d2a3e75fa3b6756 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.200441 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
I20260709 13:18:37.212996 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
I20260709 13:18:37.232518 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:37.233186 10257 tablet_replica.cc:333] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c: stopping tablet replica
I20260709 13:18:37.233744 10257 raft_consensus.cc:2243] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.234189 10257 raft_consensus.cc:2272] T 08d9237d2f2a47858badc3c4ae5ad260 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.236665 10257 tablet_replica.cc:333] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c: stopping tablet replica
I20260709 13:18:37.237185 10257 raft_consensus.cc:2243] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.237603 10257 raft_consensus.cc:2272] T 47e7a8d209014afc8205521c6ca3392f P 32b9b02685874c85b6bc1b25ccc45b1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.240110 10257 tablet_replica.cc:333] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c: stopping tablet replica
I20260709 13:18:37.240619 10257 raft_consensus.cc:2243] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:37.241472 10257 raft_consensus.cc:2272] T 772f765b5d4d43ce91bae805131a5725 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.243997 10257 tablet_replica.cc:333] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c: stopping tablet replica
I20260709 13:18:37.244488 10257 raft_consensus.cc:2243] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.244993 10257 raft_consensus.cc:2272] T 6834d4df28a04d7892f4510fb1903822 P 32b9b02685874c85b6bc1b25ccc45b1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.267351 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
I20260709 13:18:37.279887 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
I20260709 13:18:37.300518 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:37.301175 10257 tablet_replica.cc:333] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6: stopping tablet replica
I20260709 13:18:37.301783 10257 raft_consensus.cc:2243] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.302222 10257 raft_consensus.cc:2272] T 47e7a8d209014afc8205521c6ca3392f P 463c3b1e2fbb4b69af07fda5335853c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.304387 10257 tablet_replica.cc:333] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: stopping tablet replica
I20260709 13:18:37.304879 10257 raft_consensus.cc:2243] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:37.305433 10257 pending_rounds.cc:70] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: Trying to abort 1 pending ops.
I20260709 13:18:37.305601 10257 pending_rounds.cc:77] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305638367176450048 op_type: NO_OP noop_request { }
I20260709 13:18:37.305836 10257 raft_consensus.cc:2889] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:18:37.306085 10257 raft_consensus.cc:2272] T 08d9237d2f2a47858badc3c4ae5ad260 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.308264 10257 tablet_replica.cc:333] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6: stopping tablet replica
I20260709 13:18:37.308748 10257 raft_consensus.cc:2243] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.309134 10257 raft_consensus.cc:2272] T 6834d4df28a04d7892f4510fb1903822 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.311245 10257 tablet_replica.cc:333] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6: stopping tablet replica
I20260709 13:18:37.311731 10257 raft_consensus.cc:2243] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:37.312106 10257 raft_consensus.cc:2272] T 772f765b5d4d43ce91bae805131a5725 P 463c3b1e2fbb4b69af07fda5335853c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.334287 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
I20260709 13:18:37.346062 10257 master.cc:562] Master@127.10.4.126:41137 shutting down...
I20260709 13:18:37.358692 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:37.359182 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:37.359483 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P 928a02b05f3844c29c30005038559871: stopping tablet replica
I20260709 13:18:37.377315 10257 master.cc:584] Master@127.10.4.126:41137 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2237 ms)
[----------] 6 tests from LeaderRebalancerTest (282295 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:18:37.403899 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:45777
I20260709 13:18:37.405007 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.409556 20179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.410475 20180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.411355 20182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.413261 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:37.414007 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.414179 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.414320 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603117414308 us; error 0 us; skew 500 ppm
I20260709 13:18:37.414845 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.416954 10257 webserver.cc:533] Webserver started at http://127.10.4.126:38145/ using document root <none> and password file <none>
I20260709 13:18:37.417387 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.417546 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.417776 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.418761 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "98e48a1d079940ab9965431b63a5bbaa"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.422799 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:18:37.425880 20187 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.426527 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:18:37.426772 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "98e48a1d079940ab9965431b63a5bbaa"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.427021 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.438098 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.439051 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.472204 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:45777
I20260709 13:18:37.472316 20238 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:45777 every 8 connection(s)
I20260709 13:18:37.475716 20239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.486145 20239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa: Bootstrap starting.
I20260709 13:18:37.490450 20239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:37.494266 20239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa: No bootstrap required, opened a new log
I20260709 13:18:37.496326 20239 raft_consensus.cc:359] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e48a1d079940ab9965431b63a5bbaa" member_type: VOTER }
I20260709 13:18:37.496704 20239 raft_consensus.cc:385] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:37.496879 20239 raft_consensus.cc:740] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e48a1d079940ab9965431b63a5bbaa, State: Initialized, Role: FOLLOWER
I20260709 13:18:37.497453 20239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e48a1d079940ab9965431b63a5bbaa" member_type: VOTER }
I20260709 13:18:37.498034 20239 raft_consensus.cc:399] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:18:37.498234 20239 raft_consensus.cc:493] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:18:37.498449 20239 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:37.503564 20239 raft_consensus.cc:515] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e48a1d079940ab9965431b63a5bbaa" member_type: VOTER }
I20260709 13:18:37.504092 20239 leader_election.cc:304] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98e48a1d079940ab9965431b63a5bbaa; no voters: 
I20260709 13:18:37.505163 20239 leader_election.cc:290] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:18:37.505509 20242 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:37.506949 20242 raft_consensus.cc:697] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 1 LEADER]: Becoming Leader. State: Replica: 98e48a1d079940ab9965431b63a5bbaa, State: Running, Role: LEADER
I20260709 13:18:37.507629 20242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e48a1d079940ab9965431b63a5bbaa" member_type: VOTER }
I20260709 13:18:37.507876 20239 sys_catalog.cc:565] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:18:37.510511 20244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98e48a1d079940ab9965431b63a5bbaa. Latest consensus state: current_term: 1 leader_uuid: "98e48a1d079940ab9965431b63a5bbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e48a1d079940ab9965431b63a5bbaa" member_type: VOTER } }
I20260709 13:18:37.510547 20243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98e48a1d079940ab9965431b63a5bbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e48a1d079940ab9965431b63a5bbaa" member_type: VOTER } }
I20260709 13:18:37.511348 20244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:37.511394 20243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:37.514024 20248 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:18:37.518807 20248 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:18:37.522401 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:18:37.527021 20248 catalog_manager.cc:1383] Generated new cluster ID: 83e7e05e8d9f441d922048c57ae42c4b
I20260709 13:18:37.527253 20248 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:18:37.543115 20248 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:18:37.544531 20248 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:18:37.562592 20248 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa: Generated new TSK 0
I20260709 13:18:37.563176 20248 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:18:37.588783 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.594630 20260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.595633 20261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.597214 20263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.597366 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:37.598279 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.598455 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.598605 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603117598588 us; error 0 us; skew 500 ppm
I20260709 13:18:37.599126 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.601334 10257 webserver.cc:533] Webserver started at http://127.10.4.65:35343/ using document root <none> and password file <none>
I20260709 13:18:37.601819 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.601987 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.602221 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.603237 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "ec3890a9489c444385a32dde2824973f"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.607476 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:18:37.610535 20268 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.611186 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:37.611481 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "ec3890a9489c444385a32dde2824973f"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.611750 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.634467 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.635555 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.636901 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:37.639021 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:37.639209 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.639433 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:37.639582 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.675657 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:44339
I20260709 13:18:37.675761 20330 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:44339 every 8 connection(s)
I20260709 13:18:37.680251 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.687237 20335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.688206 20336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.690534 10257 server_base.cc:1061] running on GCE node
W20260709 13:18:37.690490 20338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.693109 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.693396 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.693578 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603117693556 us; error 0 us; skew 500 ppm
I20260709 13:18:37.694094 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.694139 20331 heartbeater.cc:344] Connected to a master server at 127.10.4.126:45777
I20260709 13:18:37.694602 20331 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:37.695528 20331 heartbeater.cc:507] Master 127.10.4.126:45777 requested a full tablet report, sending...
I20260709 13:18:37.696957 10257 webserver.cc:533] Webserver started at http://127.10.4.66:43165/ using document root <none> and password file <none>
I20260709 13:18:37.697570 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.697896 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.697718 20204 ts_manager.cc:194] Registered new tserver with Master: ec3890a9489c444385a32dde2824973f (127.10.4.65:44339)
I20260709 13:18:37.698426 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.699632 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "4cd696c5af034cc098882be56fff7076"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.699836 20204 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42990
I20260709 13:18:37.704468 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:37.707806 20343 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.708498 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:37.708770 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "4cd696c5af034cc098882be56fff7076"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.709012 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.722790 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.723923 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.725346 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:37.727612 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:37.727805 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.728034 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:37.728174 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.766645 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:39101
I20260709 13:18:37.766774 20405 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:39101 every 8 connection(s)
I20260709 13:18:37.771792 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.778610 20409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.780062 20410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.781582 20412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.782219 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:37.783382 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.783668 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.783861 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603117783843 us; error 0 us; skew 500 ppm
I20260709 13:18:37.784389 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.784420 20406 heartbeater.cc:344] Connected to a master server at 127.10.4.126:45777
I20260709 13:18:37.784912 20406 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:37.785821 20406 heartbeater.cc:507] Master 127.10.4.126:45777 requested a full tablet report, sending...
I20260709 13:18:37.786873 10257 webserver.cc:533] Webserver started at http://127.10.4.67:34965/ using document root <none> and password file <none>
I20260709 13:18:37.787436 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.787626 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.787873 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.788242 20204 ts_manager.cc:194] Registered new tserver with Master: 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:37.788998 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "4c7e3a8fef9c47b5b879adb450333905"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.790201 20204 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43006
I20260709 13:18:37.793527 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:18:37.797420 20417 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.798280 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:18:37.798657 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "4c7e3a8fef9c47b5b879adb450333905"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-gkm9"
I20260709 13:18:37.798961 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.807788 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.808876 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.810424 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:37.812846 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:37.813069 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.813339 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:37.813531 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.853614 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:33527
I20260709 13:18:37.853719 20479 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:33527 every 8 connection(s)
I20260709 13:18:37.868249 20480 heartbeater.cc:344] Connected to a master server at 127.10.4.126:45777
I20260709 13:18:37.868742 20480 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:37.869532 20480 heartbeater.cc:507] Master 127.10.4.126:45777 requested a full tablet report, sending...
I20260709 13:18:37.871904 20204 ts_manager.cc:194] Registered new tserver with Master: 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:37.872047 10257 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015310509s
I20260709 13:18:37.873456 20204 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43020
I20260709 13:18:37.900496 20204 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43036:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:37.903069 20204 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:37.976233 20445 tablet_service.cc:1511] Processing CreateTablet for tablet b2b5471415d545cd8e6fc1c4f2711a88 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:37.977576 20371 tablet_service.cc:1511] Processing CreateTablet for tablet b2b5471415d545cd8e6fc1c4f2711a88 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:37.977624 20445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2b5471415d545cd8e6fc1c4f2711a88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.979005 20371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2b5471415d545cd8e6fc1c4f2711a88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.979130 20442 tablet_service.cc:1511] Processing CreateTablet for tablet b6ac007579544fe196e9a79834912e9e (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:37.979696 20443 tablet_service.cc:1511] Processing CreateTablet for tablet e14b49abcadf494fb27aca35637c2bd6 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:37.980634 20442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ac007579544fe196e9a79834912e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.980451 20370 tablet_service.cc:1511] Processing CreateTablet for tablet c0427a1799c24e9cb0546a29f73b1714 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:37.981463 20443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14b49abcadf494fb27aca35637c2bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.982651 20369 tablet_service.cc:1511] Processing CreateTablet for tablet e14b49abcadf494fb27aca35637c2bd6 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:37.978644 20444 tablet_service.cc:1511] Processing CreateTablet for tablet c0427a1799c24e9cb0546a29f73b1714 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:37.984412 20295 tablet_service.cc:1511] Processing CreateTablet for tablet c0427a1799c24e9cb0546a29f73b1714 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:37.985752 20295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0427a1799c24e9cb0546a29f73b1714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.986753 20296 tablet_service.cc:1511] Processing CreateTablet for tablet b2b5471415d545cd8e6fc1c4f2711a88 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:37.988030 20296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2b5471415d545cd8e6fc1c4f2711a88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.989049 20293 tablet_service.cc:1511] Processing CreateTablet for tablet b6ac007579544fe196e9a79834912e9e (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:37.984668 20444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0427a1799c24e9cb0546a29f73b1714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.990391 20293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ac007579544fe196e9a79834912e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.988063 20294 tablet_service.cc:1511] Processing CreateTablet for tablet e14b49abcadf494fb27aca35637c2bd6 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:37.984759 20368 tablet_service.cc:1511] Processing CreateTablet for tablet b6ac007579544fe196e9a79834912e9e (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:37.995253 20294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14b49abcadf494fb27aca35637c2bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.995651 20369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14b49abcadf494fb27aca35637c2bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.998358 20368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ac007579544fe196e9a79834912e9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.999747 20370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0427a1799c24e9cb0546a29f73b1714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.043825 20296 tablet_service.cc:1511] Processing CreateTablet for tablet ab534676680e4b1184abab27ddb6752c (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:38.046116 20496 tablet_bootstrap.cc:492] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.046741 20296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab534676680e4b1184abab27ddb6752c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.048656 20495 tablet_bootstrap.cc:492] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.052958 20496 tablet_bootstrap.cc:654] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.056846 20294 tablet_service.cc:1511] Processing CreateTablet for tablet a87f179730e34788b5ebbcebdd11741e (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:38.059123 20443 tablet_service.cc:1511] Processing CreateTablet for tablet ab534676680e4b1184abab27ddb6752c (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:38.060506 20443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab534676680e4b1184abab27ddb6752c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.064430 20371 tablet_service.cc:1511] Processing CreateTablet for tablet ab534676680e4b1184abab27ddb6752c (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:38.065791 20371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab534676680e4b1184abab27ddb6752c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.076582 20497 tablet_bootstrap.cc:492] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.082490 20495 tablet_bootstrap.cc:654] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.085075 20371 tablet_service.cc:1511] Processing CreateTablet for tablet a87f179730e34788b5ebbcebdd11741e (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:38.086380 20371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87f179730e34788b5ebbcebdd11741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.089437 20442 tablet_service.cc:1511] Processing CreateTablet for tablet a87f179730e34788b5ebbcebdd11741e (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:38.090751 20442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87f179730e34788b5ebbcebdd11741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.090821 20295 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd187256e1c4ee3a19a1f4ee62d3dea (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:38.094770 20497 tablet_bootstrap.cc:654] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.098214 20368 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd187256e1c4ee3a19a1f4ee62d3dea (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:38.101910 20443 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd187256e1c4ee3a19a1f4ee62d3dea (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:38.103621 20443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd187256e1c4ee3a19a1f4ee62d3dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.103788 20293 tablet_service.cc:1511] Processing CreateTablet for tablet 259855087e02457cb10e70f2808110b8 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:38.103621 20368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd187256e1c4ee3a19a1f4ee62d3dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.106986 20370 tablet_service.cc:1511] Processing CreateTablet for tablet 259855087e02457cb10e70f2808110b8 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:38.111334 20445 tablet_service.cc:1511] Processing CreateTablet for tablet 259855087e02457cb10e70f2808110b8 (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:38.114526 20294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a87f179730e34788b5ebbcebdd11741e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.115042 20370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 259855087e02457cb10e70f2808110b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.116263 20445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 259855087e02457cb10e70f2808110b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.117372 20293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 259855087e02457cb10e70f2808110b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.119786 20295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd187256e1c4ee3a19a1f4ee62d3dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.133741 20292 tablet_service.cc:1511] Processing CreateTablet for tablet c7bb9c20fc774e248068eb0fb922884d (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:38.135149 20292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7bb9c20fc774e248068eb0fb922884d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.152529 20369 tablet_service.cc:1511] Processing CreateTablet for tablet c7bb9c20fc774e248068eb0fb922884d (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:38.153963 20369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7bb9c20fc774e248068eb0fb922884d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.157269 20495 tablet_bootstrap.cc:492] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.157886 20495 ts_tablet_manager.cc:1403] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.109s	user 0.019s	sys 0.011s
I20260709 13:18:38.160789 20495 raft_consensus.cc:359] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.161499 20495 raft_consensus.cc:385] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.161791 20495 raft_consensus.cc:740] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.175196 20497 tablet_bootstrap.cc:492] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.175773 20497 ts_tablet_manager.cc:1403] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.099s	user 0.022s	sys 0.020s
I20260709 13:18:38.172600 20495 consensus_queue.cc:260] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.176239 20444 tablet_service.cc:1511] Processing CreateTablet for tablet c7bb9c20fc774e248068eb0fb922884d (DEFAULT_TABLE table=test-workload [id=9b633fc34f544505bf0415e5b79e7600]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:38.178666 20444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7bb9c20fc774e248068eb0fb922884d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.178795 20497 raft_consensus.cc:359] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.180492 20497 raft_consensus.cc:385] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.180780 20497 raft_consensus.cc:740] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.181622 20497 consensus_queue.cc:260] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.183518 20496 tablet_bootstrap.cc:492] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.184028 20496 ts_tablet_manager.cc:1403] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.138s	user 0.036s	sys 0.046s
I20260709 13:18:38.186920 20496 raft_consensus.cc:359] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.187678 20496 raft_consensus.cc:385] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.187965 20496 raft_consensus.cc:740] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.188699 20496 consensus_queue.cc:260] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.196691 20331 heartbeater.cc:499] Master 127.10.4.126:45777 was elected leader, sending a full tablet report...
I20260709 13:18:38.197263 20495 ts_tablet_manager.cc:1434] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.009s
I20260709 13:18:38.198560 20495 tablet_bootstrap.cc:492] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.205269 20495 tablet_bootstrap.cc:654] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.219822 20496 ts_tablet_manager.cc:1434] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.035s	user 0.012s	sys 0.021s
I20260709 13:18:38.220541 20406 heartbeater.cc:499] Master 127.10.4.126:45777 was elected leader, sending a full tablet report...
I20260709 13:18:38.220996 20496 tablet_bootstrap.cc:492] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.227130 20496 tablet_bootstrap.cc:654] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.242720 20497 ts_tablet_manager.cc:1434] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.067s	user 0.015s	sys 0.037s
I20260709 13:18:38.243189 20480 heartbeater.cc:499] Master 127.10.4.126:45777 was elected leader, sending a full tablet report...
I20260709 13:18:38.244094 20497 tablet_bootstrap.cc:492] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.253227 20506 raft_consensus.cc:493] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.253834 20506 raft_consensus.cc:515] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.258883 20497 tablet_bootstrap.cc:654] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.274111 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
W20260709 13:18:38.276342 20418 leader_election.cc:343] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.277410 20506 leader_election.cc:290] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.279819 20495 tablet_bootstrap.cc:492] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.280736 20495 ts_tablet_manager.cc:1403] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.082s	user 0.038s	sys 0.040s
I20260709 13:18:38.285049 20495 raft_consensus.cc:359] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.285912 20495 raft_consensus.cc:385] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.286263 20495 raft_consensus.cc:740] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.287051 20496 tablet_bootstrap.cc:492] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.287704 20496 ts_tablet_manager.cc:1403] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.067s	user 0.039s	sys 0.026s
I20260709 13:18:38.290596 20496 raft_consensus.cc:359] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.291306 20496 raft_consensus.cc:385] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.287115 20495 consensus_queue.cc:260] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.291627 20496 raft_consensus.cc:740] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.292389 20496 consensus_queue.cc:260] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.294436 20496 ts_tablet_manager.cc:1434] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:18:38.295032 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.295428 20496 tablet_bootstrap.cc:492] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.295761 20381 raft_consensus.cc:2468] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:38.296763 20495 ts_tablet_manager.cc:1434] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:18:38.297158 20419 leader_election.cc:304] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: ec3890a9489c444385a32dde2824973f
I20260709 13:18:38.297902 20495 tablet_bootstrap.cc:492] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.300441 20497 tablet_bootstrap.cc:492] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.300593 20506 raft_consensus.cc:2804] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.301076 20506 raft_consensus.cc:493] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.301167 20497 ts_tablet_manager.cc:1403] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.006s
I20260709 13:18:38.301457 20506 raft_consensus.cc:3060] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.302390 20496 tablet_bootstrap.cc:654] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.304059 20497 raft_consensus.cc:359] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.304756 20495 tablet_bootstrap.cc:654] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.304776 20497 raft_consensus.cc:385] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.305461 20497 raft_consensus.cc:740] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.306242 20497 consensus_queue.cc:260] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.308674 20497 ts_tablet_manager.cc:1434] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:38.309582 20497 tablet_bootstrap.cc:492] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.316519 20497 tablet_bootstrap.cc:654] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.318161 20496 tablet_bootstrap.cc:492] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.318516 20501 raft_consensus.cc:493] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.318888 20496 ts_tablet_manager.cc:1403] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.010s
I20260709 13:18:38.319094 20501 raft_consensus.cc:515] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.321725 20501 leader_election.cc:290] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.322252 20496 raft_consensus.cc:359] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.323055 20496 raft_consensus.cc:385] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.323364 20496 raft_consensus.cc:740] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.324349 20496 consensus_queue.cc:260] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.326946 20496 ts_tablet_manager.cc:1434] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:18:38.328246 20496 tablet_bootstrap.cc:492] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.333081 20497 tablet_bootstrap.cc:492] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.333660 20497 ts_tablet_manager.cc:1403] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:18:38.318606 20506 raft_consensus.cc:515] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.334395 20496 tablet_bootstrap.cc:654] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.336553 20506 leader_election.cc:290] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.336982 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:38.337621 20381 raft_consensus.cc:3060] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.336465 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
W20260709 13:18:38.340502 20418 leader_election.cc:343] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.343859 20497 raft_consensus.cc:359] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.344650 20497 raft_consensus.cc:385] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.344924 20497 raft_consensus.cc:740] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.345196 20503 raft_consensus.cc:493] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.345695 20503 raft_consensus.cc:515] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.345662 20496 tablet_bootstrap.cc:492] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.348024 20496 ts_tablet_manager.cc:1403] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.006s
I20260709 13:18:38.350250 20495 tablet_bootstrap.cc:492] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.350284 20381 raft_consensus.cc:2468] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:38.351014 20495 ts_tablet_manager.cc:1403] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.053s	user 0.028s	sys 0.011s
I20260709 13:18:38.353372 20496 raft_consensus.cc:359] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.354265 20495 raft_consensus.cc:359] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.354610 20496 raft_consensus.cc:385] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.355136 20496 raft_consensus.cc:740] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.355911 20497 consensus_queue.cc:260] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.356030 20496 consensus_queue.cc:260] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.361342 20503 leader_election.cc:290] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.364359 20497 ts_tablet_manager.cc:1434] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.004s
I20260709 13:18:38.365549 20497 tablet_bootstrap.cc:492] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.366748 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b5471415d545cd8e6fc1c4f2711a88" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.367488 20381 raft_consensus.cc:2468] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:38.368835 20270 leader_election.cc:304] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.369659 20501 raft_consensus.cc:2804] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.370013 20501 raft_consensus.cc:493] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.370321 20501 raft_consensus.cc:3060] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.371430 20497 tablet_bootstrap.cc:654] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.355028 20495 raft_consensus.cc:385] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.374492 20495 raft_consensus.cc:740] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.375301 20495 consensus_queue.cc:260] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.378563 20501 raft_consensus.cc:515] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.382051 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b5471415d545cd8e6fc1c4f2711a88" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:38.382679 20381 raft_consensus.cc:3060] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.390898 20381 raft_consensus.cc:2468] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:38.392992 20419 leader_election.cc:304] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: ec3890a9489c444385a32dde2824973f
I20260709 13:18:38.394510 20506 raft_consensus.cc:2804] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.395069 20506 raft_consensus.cc:697] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:38.395993 20506 consensus_queue.cc:237] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.398440 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b5471415d545cd8e6fc1c4f2711a88" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:38.399227 20455 raft_consensus.cc:2468] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:38.401384 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2b5471415d545cd8e6fc1c4f2711a88" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:38.401988 20455 raft_consensus.cc:3060] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.411180 20501 leader_election.cc:290] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.417008 20203 catalog_manager.cc:5697] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:38.423898 20495 ts_tablet_manager.cc:1434] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.073s	user 0.011s	sys 0.023s
I20260709 13:18:38.425328 20495 tablet_bootstrap.cc:492] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.428365 20270 leader_election.cc:304] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.429355 20501 raft_consensus.cc:2804] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.431358 20495 tablet_bootstrap.cc:654] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.431807 20455 raft_consensus.cc:2468] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:38.431363 20501 raft_consensus.cc:697] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:38.437546 20501 consensus_queue.cc:237] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } }
I20260709 13:18:38.440109 20496 ts_tablet_manager.cc:1434] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.092s	user 0.019s	sys 0.048s
I20260709 13:18:38.439849 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:38.443128 20496 tablet_bootstrap.cc:492] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.443301 20497 tablet_bootstrap.cc:492] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.443951 20497 ts_tablet_manager.cc:1403] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.020s
I20260709 13:18:38.443272 20455 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:38.446972 20345 leader_election.cc:304] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:38.443553 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:38.448421 20306 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:38.450629 20496 tablet_bootstrap.cc:654] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.447953 20497 raft_consensus.cc:359] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.451589 20497 raft_consensus.cc:385] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.451920 20497 raft_consensus.cc:740] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.452493 20517 raft_consensus.cc:2804] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.452930 20517 raft_consensus.cc:493] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.453313 20517 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.452968 20497 consensus_queue.cc:260] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.455631 20497 ts_tablet_manager.cc:1434] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.006s
I20260709 13:18:38.456751 20497 tablet_bootstrap.cc:492] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.450493 20495 tablet_bootstrap.cc:492] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.462288 20495 ts_tablet_manager.cc:1403] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.004s
I20260709 13:18:38.465016 20202 catalog_manager.cc:5697] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 0 to 1, leader changed from <none> to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 1 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:38.467764 20497 tablet_bootstrap.cc:654] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.469877 20495 raft_consensus.cc:359] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.470656 20495 raft_consensus.cc:385] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.470696 20517 raft_consensus.cc:515] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.473757 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:38.474334 20306 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.475569 20496 tablet_bootstrap.cc:492] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.476225 20496 ts_tablet_manager.cc:1403] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.007s
I20260709 13:18:38.471091 20495 raft_consensus.cc:740] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.480109 20496 raft_consensus.cc:359] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.480877 20306 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:38.481043 20496 raft_consensus.cc:385] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.481559 20496 raft_consensus.cc:740] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.482121 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:38.482299 20344 leader_election.cc:304] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.482766 20455 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.483284 20503 raft_consensus.cc:2804] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.482434 20496 consensus_queue.cc:260] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.480180 20495 consensus_queue.cc:260] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.484851 20517 leader_election.cc:290] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.486778 20497 tablet_bootstrap.cc:492] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.487385 20497 ts_tablet_manager.cc:1403] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.012s
I20260709 13:18:38.487546 20495 ts_tablet_manager.cc:1434] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:18:38.488601 20495 tablet_bootstrap.cc:492] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.490593 20497 raft_consensus.cc:359] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.493430 20497 raft_consensus.cc:385] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.494431 20497 raft_consensus.cc:740] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.495761 20495 tablet_bootstrap.cc:654] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.496759 20503 raft_consensus.cc:697] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:38.497870 20503 consensus_queue.cc:237] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.499015 20496 ts_tablet_manager.cc:1434] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.003s
I20260709 13:18:38.500630 20496 tablet_bootstrap.cc:492] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.496441 20497 consensus_queue.cc:260] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.507367 20455 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:38.509775 20497 ts_tablet_manager.cc:1434] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.003s
I20260709 13:18:38.510758 20496 tablet_bootstrap.cc:654] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.513381 20495 tablet_bootstrap.cc:492] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.514038 20497 tablet_bootstrap.cc:492] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.514575 20495 ts_tablet_manager.cc:1403] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.012s
I20260709 13:18:38.516374 20204 catalog_manager.cc:5697] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:38.519058 20252 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:38.520152 20495 raft_consensus.cc:359] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.521039 20495 raft_consensus.cc:385] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.523916 20495 raft_consensus.cc:740] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.527946 20495 consensus_queue.cc:260] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.529789 20497 tablet_bootstrap.cc:654] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.530380 20495 ts_tablet_manager.cc:1434] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:18:38.531481 20495 tablet_bootstrap.cc:492] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.537741 20495 tablet_bootstrap.cc:654] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.542564 20497 tablet_bootstrap.cc:492] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.543244 20497 ts_tablet_manager.cc:1403] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.008s
I20260709 13:18:38.546553 20497 raft_consensus.cc:359] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.547525 20497 raft_consensus.cc:385] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.547886 20497 raft_consensus.cc:740] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.548893 20497 consensus_queue.cc:260] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.551319 20497 ts_tablet_manager.cc:1434] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:38.552289 20497 tablet_bootstrap.cc:492] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.553541 20496 tablet_bootstrap.cc:492] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.554098 20496 ts_tablet_manager.cc:1403] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.054s	user 0.024s	sys 0.019s
I20260709 13:18:38.557174 20496 raft_consensus.cc:359] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.557886 20496 raft_consensus.cc:385] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.558141 20496 raft_consensus.cc:740] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.558604 20497 tablet_bootstrap.cc:654] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.558971 20496 consensus_queue.cc:260] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.561264 20506 raft_consensus.cc:493] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.561794 20506 raft_consensus.cc:515] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.562280 20495 tablet_bootstrap.cc:492] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.562918 20495 ts_tablet_manager.cc:1403] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.015s
I20260709 13:18:38.563980 20506 leader_election.cc:290] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.566171 20496 ts_tablet_manager.cc:1434] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:18:38.567050 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:38.567414 20496 tablet_bootstrap.cc:492] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.568275 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.568969 20381 raft_consensus.cc:2468] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:38.569242 20495 raft_consensus.cc:359] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.570048 20495 raft_consensus.cc:385] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.570236 20419 leader_election.cc:304] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:38.570429 20495 raft_consensus.cc:740] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.571223 20497 tablet_bootstrap.cc:492] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
W20260709 13:18:38.571600 20418 leader_election.cc:343] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.571781 20497 ts_tablet_manager.cc:1403] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:18:38.571393 20495 consensus_queue.cc:260] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.573290 20506 raft_consensus.cc:2804] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.573750 20506 raft_consensus.cc:493] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.574098 20506 raft_consensus.cc:3060] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.584214 20495 ts_tablet_manager.cc:1434] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 13:18:38.584862 20496 tablet_bootstrap.cc:654] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.584918 20497 raft_consensus.cc:359] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.585785 20497 raft_consensus.cc:385] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.586052 20497 raft_consensus.cc:740] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.586900 20497 consensus_queue.cc:260] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.588162 20495 tablet_bootstrap.cc:492] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.589490 20497 ts_tablet_manager.cc:1434] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.005s
I20260709 13:18:38.590626 20497 tablet_bootstrap.cc:492] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.594202 20502 raft_consensus.cc:493] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.594784 20502 raft_consensus.cc:515] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.597126 20502 leader_election.cc:290] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.597497 20495 tablet_bootstrap.cc:654] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.599629 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:38.600296 20306 raft_consensus.cc:2468] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:38.601547 20418 leader_election.cc:304] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.602449 20502 raft_consensus.cc:2804] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.602831 20502 raft_consensus.cc:493] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.603204 20502 raft_consensus.cc:3060] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.602936 20497 tablet_bootstrap.cc:654] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.605526 20496 tablet_bootstrap.cc:492] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.606065 20496 ts_tablet_manager.cc:1403] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.039s	user 0.006s	sys 0.019s
I20260709 13:18:38.609196 20496 raft_consensus.cc:359] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.609930 20496 raft_consensus.cc:385] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.610198 20496 raft_consensus.cc:740] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.611279 20495 tablet_bootstrap.cc:492] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.611100 20496 consensus_queue.cc:260] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.611922 20495 ts_tablet_manager.cc:1403] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 13:18:38.597836 20506 raft_consensus.cc:515] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.598623 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.613363 20381 raft_consensus.cc:2468] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:38.615229 20495 raft_consensus.cc:359] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.616093 20495 raft_consensus.cc:385] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.616269 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:38.616451 20495 raft_consensus.cc:740] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.616930 20496 ts_tablet_manager.cc:1434] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
W20260709 13:18:38.617424 20418 leader_election.cc:343] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.618003 20496 tablet_bootstrap.cc:492] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.617683 20495 consensus_queue.cc:260] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.618726 20497 tablet_bootstrap.cc:492] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.619200 20497 ts_tablet_manager.cc:1403] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.004s
I20260709 13:18:38.619789 20502 raft_consensus.cc:515] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.621990 20497 raft_consensus.cc:359] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.622427 20495 ts_tablet_manager.cc:1434] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 13:18:38.622725 20497 raft_consensus.cc:385] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.623173 20497 raft_consensus.cc:740] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.623816 20495 tablet_bootstrap.cc:492] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.624013 20497 consensus_queue.cc:260] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.624791 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:38.625029 20496 tablet_bootstrap.cc:654] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.625430 20381 raft_consensus.cc:3060] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.625650 20501 raft_consensus.cc:493] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.626188 20501 raft_consensus.cc:515] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.628188 20501 leader_election.cc:290] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.629846 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.629947 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
W20260709 13:18:38.631476 20270 leader_election.cc:343] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:38.632174 20270 leader_election.cc:343] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.632565 20270 leader_election.cc:304] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec3890a9489c444385a32dde2824973f; no voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076
I20260709 13:18:38.633461 20497 ts_tablet_manager.cc:1434] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.005s
I20260709 13:18:38.633623 20501 raft_consensus.cc:2749] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:38.634475 20497 tablet_bootstrap.cc:492] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:38.637308 20495 tablet_bootstrap.cc:654] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.640288 20506 leader_election.cc:290] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.642103 20496 tablet_bootstrap.cc:492] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.642587 20496 ts_tablet_manager.cc:1403] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.006s
I20260709 13:18:38.643080 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:38.643788 20306 raft_consensus.cc:3060] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.646446 20497 tablet_bootstrap.cc:654] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.652680 20306 raft_consensus.cc:2468] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:38.653475 20496 raft_consensus.cc:359] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.654160 20418 leader_election.cc:304] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.654309 20496 raft_consensus.cc:385] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.655081 20496 raft_consensus.cc:740] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.656322 20496 consensus_queue.cc:260] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.660130 20506 raft_consensus.cc:2804] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.661568 20506 raft_consensus.cc:697] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:38.662541 20506 consensus_queue.cc:237] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.666860 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:38.667044 20496 ts_tablet_manager.cc:1434] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 13:18:38.667603 20380 raft_consensus.cc:3060] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.675581 20502 leader_election.cc:290] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.676575 20496 tablet_bootstrap.cc:492] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:38.678428 20380 raft_consensus.cc:2468] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:38.632865 20381 raft_consensus.cc:2468] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:38.681819 20419 leader_election.cc:304] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: ec3890a9489c444385a32dde2824973f
I20260709 13:18:38.683305 20502 raft_consensus.cc:2804] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.684072 20502 raft_consensus.cc:697] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:38.684978 20502 consensus_queue.cc:237] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.687268 20496 tablet_bootstrap.cc:654] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.691998 20503 raft_consensus.cc:493] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.692575 20503 raft_consensus.cc:515] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.696411 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:38.696612 20503 leader_election.cc:290] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.697050 20455 raft_consensus.cc:2468] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:38.697525 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:38.698251 20306 raft_consensus.cc:2468] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:38.698463 20345 leader_election.cc:304] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:38.696866 20204 catalog_manager.cc:5697] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:38.699414 20503 raft_consensus.cc:2804] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.699836 20503 raft_consensus.cc:493] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.700291 20503 raft_consensus.cc:3060] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.701386 20497 tablet_bootstrap.cc:492] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:38.701992 20497 ts_tablet_manager.cc:1403] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.068s	user 0.020s	sys 0.037s
I20260709 13:18:38.704984 20497 raft_consensus.cc:359] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.705703 20497 raft_consensus.cc:385] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.705966 20497 raft_consensus.cc:740] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.706790 20497 consensus_queue.cc:260] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.709004 20497 ts_tablet_manager.cc:1434] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:38.709022 20503 raft_consensus.cc:515] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.711915 20495 tablet_bootstrap.cc:492] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.712409 20495 ts_tablet_manager.cc:1403] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.089s	user 0.035s	sys 0.024s
I20260709 13:18:38.713855 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:38.714468 20306 raft_consensus.cc:3060] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.715122 20495 raft_consensus.cc:359] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.715940 20495 raft_consensus.cc:385] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.716202 20495 raft_consensus.cc:740] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.716985 20204 catalog_manager.cc:5697] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:38.718503 20496 tablet_bootstrap.cc:492] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:38.719054 20496 ts_tablet_manager.cc:1403] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.002s
I20260709 13:18:38.720517 20503 leader_election.cc:290] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.722086 20306 raft_consensus.cc:2468] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:38.721915 20496 raft_consensus.cc:359] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.722620 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:38.716950 20495 consensus_queue.cc:260] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.723304 20455 raft_consensus.cc:3060] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.723361 20344 leader_election.cc:304] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.725062 20495 ts_tablet_manager.cc:1434] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:18:38.725880 20495 tablet_bootstrap.cc:492] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:38.722667 20496 raft_consensus.cc:385] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.728036 20496 raft_consensus.cc:740] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.728813 20496 consensus_queue.cc:260] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.730926 20496 ts_tablet_manager.cc:1434] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 13:18:38.731915 20526 raft_consensus.cc:2804] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.732023 20455 raft_consensus.cc:2468] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:38.732625 20526 raft_consensus.cc:697] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:38.733723 20495 tablet_bootstrap.cc:654] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.733566 20526 consensus_queue.cc:237] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.741099 20495 tablet_bootstrap.cc:492] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:38.741747 20495 ts_tablet_manager.cc:1403] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.004s
I20260709 13:18:38.744663 20495 raft_consensus.cc:359] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.745235 20495 raft_consensus.cc:385] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.745438 20495 raft_consensus.cc:740] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.746129 20495 consensus_queue.cc:260] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.748432 20495 ts_tablet_manager.cc:1434] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:38.748631 20204 catalog_manager.cc:5697] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:38.752080 20526 raft_consensus.cc:493] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.752772 20526 raft_consensus.cc:515] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.754626 20526 leader_election.cc:290] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.756070 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:38.756330 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:38.757014 20306 raft_consensus.cc:2468] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:38.758134 20344 leader_election.cc:304] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.758975 20526 raft_consensus.cc:2804] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.759305 20455 raft_consensus.cc:2468] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:38.759471 20526 raft_consensus.cc:493] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.760026 20526 raft_consensus.cc:3060] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.765798 20526 raft_consensus.cc:515] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.767633 20526 leader_election.cc:290] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.768451 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:38.768746 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:38.769019 20306 raft_consensus.cc:3060] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.769326 20455 raft_consensus.cc:3060] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.774339 20455 raft_consensus.cc:2468] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:38.774338 20306 raft_consensus.cc:2468] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:38.775771 20345 leader_election.cc:304] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:38.776594 20526 raft_consensus.cc:2804] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.777019 20526 raft_consensus.cc:697] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:38.777772 20526 consensus_queue.cc:237] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.782114 20502 consensus_queue.cc:1048] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:38.785320 20202 catalog_manager.cc:5697] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:38.793100 20306 raft_consensus.cc:3060] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.799758 20502 raft_consensus.cc:493] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.800312 20502 raft_consensus.cc:515] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.802126 20502 leader_election.cc:290] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.803448 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:38.803578 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.804255 20306 raft_consensus.cc:2468] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:38.804723 20380 raft_consensus.cc:2468] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:38.803920 20502 consensus_queue.cc:1048] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:18:38.806998 20418 leader_election.cc:304] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.807850 20502 raft_consensus.cc:2804] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.808218 20502 raft_consensus.cc:493] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.808537 20502 raft_consensus.cc:3060] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.817914 20501 raft_consensus.cc:493] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.818501 20501 raft_consensus.cc:515] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.818632 20502 raft_consensus.cc:515] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.820432 20501 leader_election.cc:290] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.821511 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.821978 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:38.821494 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:38.822579 20381 raft_consensus.cc:2393] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec3890a9489c444385a32dde2824973f in current term 1: Already voted for candidate 4c7e3a8fef9c47b5b879adb450333905 in this term.
I20260709 13:18:38.822862 20306 raft_consensus.cc:3060] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.823357 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:38.824033 20380 raft_consensus.cc:3060] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.824093 20270 leader_election.cc:304] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec3890a9489c444385a32dde2824973f; no voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076
I20260709 13:18:38.826138 20501 raft_consensus.cc:3060] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.831215 20306 raft_consensus.cc:2468] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:38.831521 20380 raft_consensus.cc:2468] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:38.832002 20502 leader_election.cc:290] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:38.832721 20418 leader_election.cc:304] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.833501 20501 raft_consensus.cc:2749] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:38.833539 20502 raft_consensus.cc:2804] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.834261 20502 raft_consensus.cc:697] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:38.834967 20502 consensus_queue.cc:237] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.842312 20202 catalog_manager.cc:5697] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:38.885677 20501 consensus_queue.cc:1048] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:38.897840 20501 consensus_queue.cc:1048] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:38.976630 20501 raft_consensus.cc:493] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.977113 20501 raft_consensus.cc:515] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.978848 20501 leader_election.cc:290] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.979789 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "259855087e02457cb10e70f2808110b8" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:38.979996 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "259855087e02457cb10e70f2808110b8" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:38.980451 20380 raft_consensus.cc:2468] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:38.980655 20455 raft_consensus.cc:2468] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:38.981423 20270 leader_election.cc:304] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.982143 20501 raft_consensus.cc:2804] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.982450 20501 raft_consensus.cc:493] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.982700 20501 raft_consensus.cc:3060] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.988298 20501 raft_consensus.cc:515] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:38.990180 20501 leader_election.cc:290] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:38.991137 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "259855087e02457cb10e70f2808110b8" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:38.991240 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "259855087e02457cb10e70f2808110b8" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:38.991802 20380 raft_consensus.cc:3060] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.991863 20455 raft_consensus.cc:3060] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.997296 20455 raft_consensus.cc:2468] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:38.997314 20380 raft_consensus.cc:2468] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:38.998391 20270 leader_election.cc:304] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:38.999028 20501 raft_consensus.cc:2804] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.999428 20501 raft_consensus.cc:697] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:39.000193 20501 consensus_queue.cc:237] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.005115 20526 consensus_queue.cc:1048] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.007853 20204 catalog_manager.cc:5697] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 0 to 1, leader changed from <none> to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 1 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.017082 20526 consensus_queue.cc:1048] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.112596 20455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
mode: GRACEFUL
new_leader_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:44544
I20260709 13:18:39.113229 20455 raft_consensus.cc:606] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Received request to transfer leadership to TS ec3890a9489c444385a32dde2824973f
I20260709 13:18:39.116102 20502 consensus_queue.cc:1048] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.127511 20506 consensus_queue.cc:1048] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.129884 20381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
mode: GRACEFUL
new_leader_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:48436
I20260709 13:18:39.130460 20381 raft_consensus.cc:606] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Received request to transfer leadership to TS ec3890a9489c444385a32dde2824973f
I20260709 13:18:39.137813 20536 raft_consensus.cc:993] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905: : Instructing follower ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:39.138257 20536 raft_consensus.cc:1081] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Signalling peer ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:39.138765 20455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
mode: GRACEFUL
new_leader_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:44544
I20260709 13:18:39.139355 20455 raft_consensus.cc:606] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Received request to transfer leadership to TS ec3890a9489c444385a32dde2824973f
I20260709 13:18:39.140465 20306 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:49052
I20260709 13:18:39.141110 20306 raft_consensus.cc:493] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.141490 20306 raft_consensus.cc:3060] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.144940 20455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
mode: GRACEFUL
new_leader_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:44544
I20260709 13:18:39.145455 20455 raft_consensus.cc:606] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Received request to transfer leadership to TS 4cd696c5af034cc098882be56fff7076
I20260709 13:18:39.147621 20306 raft_consensus.cc:515] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.149390 20306 leader_election.cc:290] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:39.150004 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:39.150346 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0427a1799c24e9cb0546a29f73b1714" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:39.150621 20380 raft_consensus.cc:3060] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.150949 20455 raft_consensus.cc:3055] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.151278 20455 raft_consensus.cc:740] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:39.152093 20455 consensus_queue.cc:260] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.153337 20454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
mode: GRACEFUL
new_leader_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:44544
I20260709 13:18:39.153359 20455 raft_consensus.cc:3060] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.153928 20454 raft_consensus.cc:606] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Received request to transfer leadership to TS 4cd696c5af034cc098882be56fff7076
I20260709 13:18:39.157493 20380 raft_consensus.cc:2468] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:39.158542 20270 leader_election.cc:304] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:39.159304 20501 raft_consensus.cc:2804] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.159741 20501 raft_consensus.cc:697] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:39.160528 20501 consensus_queue.cc:237] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.160871 20455 raft_consensus.cc:2468] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:39.167955 20202 catalog_manager.cc:5697] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 1 to 2, leader changed from 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67) to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 2 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.169463 20306 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "259855087e02457cb10e70f2808110b8"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
mode: GRACEFUL
new_leader_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:37288
I20260709 13:18:39.170058 20306 raft_consensus.cc:606] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Received request to transfer leadership to TS 4cd696c5af034cc098882be56fff7076
I20260709 13:18:39.175735 20380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
mode: GRACEFUL
new_leader_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:48436
I20260709 13:18:39.176398 20380 raft_consensus.cc:606] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Received request to transfer leadership to TS 4c7e3a8fef9c47b5b879adb450333905
I20260709 13:18:39.223279 20526 consensus_queue.cc:1048] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.236490 20536 raft_consensus.cc:993] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905: : Instructing follower ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:39.236958 20536 raft_consensus.cc:1081] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Signalling peer ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:39.237417 20525 consensus_queue.cc:1048] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:39.238629 20306 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:49052
I20260709 13:18:39.239283 20306 raft_consensus.cc:493] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.239682 20306 raft_consensus.cc:3060] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.248139 20306 raft_consensus.cc:515] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.250200 20506 consensus_queue.cc:1048] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:39.251228 20306 leader_election.cc:290] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:39.251880 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:39.252537 20380 raft_consensus.cc:3060] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.252480 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ac007579544fe196e9a79834912e9e" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:39.253388 20455 raft_consensus.cc:3055] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.253746 20455 raft_consensus.cc:740] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:39.254594 20455 consensus_queue.cc:260] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.255944 20455 raft_consensus.cc:3060] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.260429 20380 raft_consensus.cc:2468] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:39.261840 20270 leader_election.cc:304] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:39.262715 20501 raft_consensus.cc:2804] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.263200 20501 raft_consensus.cc:697] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:39.263378 20506 consensus_queue.cc:1048] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:39.263602 20455 raft_consensus.cc:2468] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:39.264387 20501 consensus_queue.cc:237] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.270407 20536 consensus_queue.cc:1048] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.278216 20503 consensus_queue.cc:1048] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:39.281060 20204 catalog_manager.cc:5697] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 1 to 2, leader changed from 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67) to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 2 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.286331 20536 raft_consensus.cc:993] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905: : Instructing follower 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:39.286897 20536 raft_consensus.cc:1081] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Signalling peer 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:39.289019 20502 raft_consensus.cc:993] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905: : Instructing follower 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:39.289695 20502 raft_consensus.cc:1081] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Signalling peer 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:39.290829 20379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:51670
I20260709 13:18:39.291718 20379 raft_consensus.cc:493] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.292135 20379 raft_consensus.cc:3060] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.292424 20381 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:51670
I20260709 13:18:39.293081 20381 raft_consensus.cc:493] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.293496 20381 raft_consensus.cc:3060] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.297029 20502 consensus_queue.cc:1048] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:39.305011 20379 raft_consensus.cc:515] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.307066 20503 raft_consensus.cc:993] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076: : Instructing follower 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:39.312135 20503 raft_consensus.cc:1081] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Signalling peer 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:39.314111 20381 raft_consensus.cc:515] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.317664 20454 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:44518
I20260709 13:18:39.318327 20454 raft_consensus.cc:493] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.318703 20454 raft_consensus.cc:3060] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.319034 20455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:39.319746 20455 raft_consensus.cc:3055] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.320117 20455 raft_consensus.cc:740] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:39.319921 20379 leader_election.cc:290] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:39.322166 20304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:39.322858 20304 raft_consensus.cc:3060] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.323283 20455 consensus_queue.cc:260] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.324570 20455 raft_consensus.cc:3060] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.324944 20453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd187256e1c4ee3a19a1f4ee62d3dea" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:39.325603 20453 raft_consensus.cc:3055] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.325902 20453 raft_consensus.cc:740] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:39.326721 20453 consensus_queue.cc:260] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.327929 20453 raft_consensus.cc:3060] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.328141 20381 leader_election.cc:290] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:39.329852 20503 consensus_queue.cc:1048] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.333755 20455 raft_consensus.cc:2468] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
I20260709 13:18:39.335007 20345 leader_election.cc:304] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:39.337431 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab534676680e4b1184abab27ddb6752c" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:39.337710 20454 raft_consensus.cc:515] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.338112 20306 raft_consensus.cc:3060] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.339360 20453 raft_consensus.cc:2468] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
I20260709 13:18:39.340626 20345 leader_election.cc:304] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:39.341487 20525 raft_consensus.cc:2804] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.341988 20525 raft_consensus.cc:697] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:39.342911 20304 raft_consensus.cc:2468] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
I20260709 13:18:39.342780 20525 consensus_queue.cc:237] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.344375 20306 raft_consensus.cc:2468] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
I20260709 13:18:39.317704 20380 raft_consensus.cc:1240] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Rejecting Update request from peer 4c7e3a8fef9c47b5b879adb450333905 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:39.347648 20503 raft_consensus.cc:2804] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.350756 20503 raft_consensus.cc:697] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:39.351681 20503 consensus_queue.cc:237] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.354844 20454 leader_election.cc:290] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 2 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:39.355958 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:39.355873 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7bb9c20fc774e248068eb0fb922884d" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:39.356850 20381 raft_consensus.cc:3055] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.357179 20381 raft_consensus.cc:740] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:39.357585 20305 raft_consensus.cc:3060] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.357978 20381 consensus_queue.cc:260] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.358083 20204 catalog_manager.cc:5697] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 1 to 2, leader changed from 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67) to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.359360 20381 raft_consensus.cc:3060] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.365119 20305 raft_consensus.cc:2468] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 2.
I20260709 13:18:39.366313 20418 leader_election.cc:304] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:39.366765 20381 raft_consensus.cc:2468] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 2.
I20260709 13:18:39.367241 20536 raft_consensus.cc:2804] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.367875 20536 raft_consensus.cc:697] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 2 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:39.368731 20536 consensus_queue.cc:237] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.378279 20204 catalog_manager.cc:5697] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 1 to 2, leader changed from 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67) to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.378794 20202 catalog_manager.cc:5697] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 1 to 2, leader changed from 4cd696c5af034cc098882be56fff7076 (127.10.4.66) to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:39.444290 20525 raft_consensus.cc:993] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076: : Instructing follower ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:39.444831 20503 raft_consensus.cc:1081] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Signalling peer ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:39.446442 20305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:49060
I20260709 13:18:39.447062 20305 raft_consensus.cc:493] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.447469 20305 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.454118 20305 raft_consensus.cc:515] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.454555 20519 consensus_queue.cc:1048] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:39.456422 20305 leader_election.cc:290] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:39.457592 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:39.457525 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:39.458388 20454 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.458482 20380 raft_consensus.cc:3055] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.458881 20380 raft_consensus.cc:740] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:39.459745 20380 consensus_queue.cc:260] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.461230 20380 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.463873 20553 raft_consensus.cc:993] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f: : Instructing follower 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:39.464576 20553 raft_consensus.cc:1081] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Signalling peer 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:39.467335 20454 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:39.468685 20381 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "259855087e02457cb10e70f2808110b8"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:51678
I20260709 13:18:39.469439 20381 raft_consensus.cc:493] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:39.468765 20270 leader_election.cc:304] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:39.469843 20381 raft_consensus.cc:3060] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.470466 20519 raft_consensus.cc:2804] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.470520 20380 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:39.471366 20501 consensus_queue.cc:1048] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.472716 20519 raft_consensus.cc:697] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:39.473707 20519 consensus_queue.cc:237] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.478991 20381 raft_consensus.cc:515] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.480242 20380 raft_consensus.cc:1240] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Rejecting Update request from peer ec3890a9489c444385a32dde2824973f for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:39.483601 20519 consensus_queue.cc:1059] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.483703 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "259855087e02457cb10e70f2808110b8" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:39.484743 20453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "259855087e02457cb10e70f2808110b8" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:39.483606 20381 leader_election.cc:290] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:39.488284 20453 raft_consensus.cc:3060] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.487481 20202 catalog_manager.cc:5697] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 1 to 2, leader changed from 4cd696c5af034cc098882be56fff7076 (127.10.4.66) to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 2 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.495760 20453 raft_consensus.cc:2468] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
I20260709 13:18:39.496904 20305 raft_consensus.cc:3055] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:39.496933 20345 leader_election.cc:304] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:39.497682 20305 raft_consensus.cc:740] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:39.498173 20503 raft_consensus.cc:2804] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:39.498735 20503 raft_consensus.cc:697] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:39.499127 20305 consensus_queue.cc:260] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.499590 20503 consensus_queue.cc:237] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:39.500721 20305 raft_consensus.cc:3060] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:39.508013 20305 raft_consensus.cc:2468] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
W20260709 13:18:39.508632 20553 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:18:39.507648 20202 catalog_manager.cc:5697] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 1 to 2, leader changed from ec3890a9489c444385a32dde2824973f (127.10.4.65) to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.637300 20453 raft_consensus.cc:1275] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.638533 20553 consensus_queue.cc:1048] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.647064 20381 raft_consensus.cc:1275] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.648788 20519 consensus_queue.cc:1048] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.795477 20305 raft_consensus.cc:1275] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.796660 20503 consensus_queue.cc:1048] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.799005 20453 raft_consensus.cc:1275] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.800663 20503 consensus_queue.cc:1048] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.845683 20381 raft_consensus.cc:1275] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.846956 20519 consensus_queue.cc:1048] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.847261 20453 raft_consensus.cc:1275] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.848688 20503 consensus_queue.cc:1048] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.849323 20453 raft_consensus.cc:1275] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.850544 20519 consensus_queue.cc:1048] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.859089 20305 raft_consensus.cc:1275] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.860628 20525 consensus_queue.cc:1048] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.926903 20305 raft_consensus.cc:1275] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.928072 20525 consensus_queue.cc:1048] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.935429 20454 raft_consensus.cc:1275] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.936962 20525 consensus_queue.cc:1048] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.940011 20381 raft_consensus.cc:1275] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.941221 20536 consensus_queue.cc:1048] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:39.950778 20305 raft_consensus.cc:1275] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:39.952069 20579 consensus_queue.cc:1048] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:40.092394 20454 raft_consensus.cc:1275] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:40.093621 20577 consensus_queue.cc:1048] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:40.096779 20381 raft_consensus.cc:1275] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:40.098881 20577 consensus_queue.cc:1048] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:42.202082 20202 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59046:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:42.204298 20202 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:42.253218 20295 tablet_service.cc:1511] Processing CreateTablet for tablet f281ecea50c24327b8006c8f08c72736 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:42.254504 20295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f281ecea50c24327b8006c8f08c72736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.255777 20444 tablet_service.cc:1511] Processing CreateTablet for tablet f281ecea50c24327b8006c8f08c72736 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:42.257035 20444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f281ecea50c24327b8006c8f08c72736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.259296 20371 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfc7b968a2a42dc92565aa8335ea320 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:42.260620 20371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfc7b968a2a42dc92565aa8335ea320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.254494 20369 tablet_service.cc:1511] Processing CreateTablet for tablet f281ecea50c24327b8006c8f08c72736 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:42.265282 20369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f281ecea50c24327b8006c8f08c72736. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.256714 20293 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfc7b968a2a42dc92565aa8335ea320 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:42.266423 20293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfc7b968a2a42dc92565aa8335ea320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.266709 20292 tablet_service.cc:1511] Processing CreateTablet for tablet 8db17f5622b84012a65f30d5ece1d98c (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:42.267967 20292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db17f5622b84012a65f30d5ece1d98c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.271102 20370 tablet_service.cc:1511] Processing CreateTablet for tablet 8db17f5622b84012a65f30d5ece1d98c (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:42.265103 20445 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfc7b968a2a42dc92565aa8335ea320 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:42.273387 20445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfc7b968a2a42dc92565aa8335ea320. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.272359 20296 tablet_service.cc:1511] Processing CreateTablet for tablet 964f1f17376d48538f5f1827dba39ee5 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:42.276338 20443 tablet_service.cc:1511] Processing CreateTablet for tablet 964f1f17376d48538f5f1827dba39ee5 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:42.277117 20296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964f1f17376d48538f5f1827dba39ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.277558 20443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964f1f17376d48538f5f1827dba39ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.280272 20442 tablet_service.cc:1511] Processing CreateTablet for tablet 8db17f5622b84012a65f30d5ece1d98c (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:42.281213 20442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db17f5622b84012a65f30d5ece1d98c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.271162 20367 tablet_service.cc:1511] Processing CreateTablet for tablet c8717bb64ae04786857242f5653cc8ee (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:42.273589 20368 tablet_service.cc:1511] Processing CreateTablet for tablet 964f1f17376d48538f5f1827dba39ee5 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:42.284065 20441 tablet_service.cc:1511] Processing CreateTablet for tablet c8717bb64ae04786857242f5653cc8ee (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:42.285279 20441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8717bb64ae04786857242f5653cc8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.272429 20370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db17f5622b84012a65f30d5ece1d98c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.289294 20291 tablet_service.cc:1511] Processing CreateTablet for tablet a5dc85cb531340ffa57355c8424590bc (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:42.290567 20291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5dc85cb531340ffa57355c8424590bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.290300 20294 tablet_service.cc:1511] Processing CreateTablet for tablet c8717bb64ae04786857242f5653cc8ee (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:42.291559 20294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8717bb64ae04786857242f5653cc8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.291760 20368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 964f1f17376d48538f5f1827dba39ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.292465 20290 tablet_service.cc:1511] Processing CreateTablet for tablet a258a503dee046afbebc36a2d850f541 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:42.293864 20367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8717bb64ae04786857242f5653cc8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.294947 20289 tablet_service.cc:1511] Processing CreateTablet for tablet 2e773ebd3ba0400181d5d44339c22cd1 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:42.297071 20288 tablet_service.cc:1511] Processing CreateTablet for tablet 4a144f8a86ec4b9982e6c06ac872b1fd (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:42.293797 20290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a258a503dee046afbebc36a2d850f541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.300601 20289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e773ebd3ba0400181d5d44339c22cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.302801 20366 tablet_service.cc:1511] Processing CreateTablet for tablet a5dc85cb531340ffa57355c8424590bc (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:42.303004 20439 tablet_service.cc:1511] Processing CreateTablet for tablet a258a503dee046afbebc36a2d850f541 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:42.303385 20440 tablet_service.cc:1511] Processing CreateTablet for tablet a5dc85cb531340ffa57355c8424590bc (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:42.303793 20288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a144f8a86ec4b9982e6c06ac872b1fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.304332 20439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a258a503dee046afbebc36a2d850f541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.304975 20366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5dc85cb531340ffa57355c8424590bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.305516 20440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5dc85cb531340ffa57355c8424590bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.307827 20365 tablet_service.cc:1511] Processing CreateTablet for tablet a258a503dee046afbebc36a2d850f541 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:42.308519 20438 tablet_service.cc:1511] Processing CreateTablet for tablet 2e773ebd3ba0400181d5d44339c22cd1 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:42.309062 20365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a258a503dee046afbebc36a2d850f541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.309746 20438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e773ebd3ba0400181d5d44339c22cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.312124 20364 tablet_service.cc:1511] Processing CreateTablet for tablet 2e773ebd3ba0400181d5d44339c22cd1 (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:42.313494 20364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e773ebd3ba0400181d5d44339c22cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.314991 20437 tablet_service.cc:1511] Processing CreateTablet for tablet 4a144f8a86ec4b9982e6c06ac872b1fd (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:42.315538 20363 tablet_service.cc:1511] Processing CreateTablet for tablet 4a144f8a86ec4b9982e6c06ac872b1fd (DEFAULT_TABLE table=soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:42.316381 20437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a144f8a86ec4b9982e6c06ac872b1fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.316901 20363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a144f8a86ec4b9982e6c06ac872b1fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:42.375959 20598 tablet_bootstrap.cc:492] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.403630 20597 tablet_bootstrap.cc:492] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.419773 20597 tablet_bootstrap.cc:654] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.441592 20598 tablet_bootstrap.cc:654] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.447666 20599 tablet_bootstrap.cc:492] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.450070 20597 tablet_bootstrap.cc:492] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.452279 20597 ts_tablet_manager.cc:1403] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.014s
I20260709 13:18:42.455371 20599 tablet_bootstrap.cc:654] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.456053 20597 raft_consensus.cc:359] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.456466 20598 tablet_bootstrap.cc:492] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.456652 20597 raft_consensus.cc:385] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.457042 20597 raft_consensus.cc:740] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.457119 20598 ts_tablet_manager.cc:1403] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.007s
I20260709 13:18:42.457795 20597 consensus_queue.cc:260] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.459780 20597 ts_tablet_manager.cc:1434] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:42.460843 20597 tablet_bootstrap.cc:492] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.460537 20598 raft_consensus.cc:359] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.461361 20598 raft_consensus.cc:385] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.461665 20598 raft_consensus.cc:740] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.462549 20598 consensus_queue.cc:260] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.464936 20598 ts_tablet_manager.cc:1434] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:42.466007 20598 tablet_bootstrap.cc:492] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.472966 20598 tablet_bootstrap.cc:654] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.483969 20597 tablet_bootstrap.cc:654] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.485785 20598 tablet_bootstrap.cc:492] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.486442 20598 ts_tablet_manager.cc:1403] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:18:42.489012 20599 tablet_bootstrap.cc:492] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.489291 20597 tablet_bootstrap.cc:492] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.489619 20599 ts_tablet_manager.cc:1403] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.016s
I20260709 13:18:42.489346 20598 raft_consensus.cc:359] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.489837 20597 ts_tablet_manager.cc:1403] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.000s
I20260709 13:18:42.490181 20598 raft_consensus.cc:385] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.490514 20598 raft_consensus.cc:740] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.491345 20598 consensus_queue.cc:260] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.492072 20599 raft_consensus.cc:359] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.492745 20599 raft_consensus.cc:385] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.493024 20599 raft_consensus.cc:740] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.493779 20598 ts_tablet_manager.cc:1434] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:42.493546 20597 raft_consensus.cc:359] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.494364 20597 raft_consensus.cc:385] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.493831 20599 consensus_queue.cc:260] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.494753 20597 raft_consensus.cc:740] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.495131 20598 tablet_bootstrap.cc:492] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.496899 20599 ts_tablet_manager.cc:1434] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:18:42.506577 20599 tablet_bootstrap.cc:492] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.505828 20597 consensus_queue.cc:260] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.510558 20597 ts_tablet_manager.cc:1434] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.003s
I20260709 13:18:42.511636 20597 tablet_bootstrap.cc:492] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.511724 20598 tablet_bootstrap.cc:654] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.518946 20599 tablet_bootstrap.cc:654] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.519057 20597 tablet_bootstrap.cc:654] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.523473 20598 tablet_bootstrap.cc:492] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.524029 20598 ts_tablet_manager.cc:1403] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.004s
I20260709 13:18:42.527004 20598 raft_consensus.cc:359] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.527850 20598 raft_consensus.cc:385] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.528133 20598 raft_consensus.cc:740] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.528290 20597 tablet_bootstrap.cc:492] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.529112 20597 ts_tablet_manager.cc:1403] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 13:18:42.528945 20598 consensus_queue.cc:260] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.531219 20598 ts_tablet_manager.cc:1434] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:42.532227 20598 tablet_bootstrap.cc:492] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.535023 20597 raft_consensus.cc:359] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.535830 20597 raft_consensus.cc:385] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.536022 20599 tablet_bootstrap.cc:492] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.536201 20597 raft_consensus.cc:740] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.536650 20599 ts_tablet_manager.cc:1403] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.006s
I20260709 13:18:42.537236 20597 consensus_queue.cc:260] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.538381 20598 tablet_bootstrap.cc:654] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.539752 20597 ts_tablet_manager.cc:1434] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:18:42.539749 20599 raft_consensus.cc:359] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.540720 20599 raft_consensus.cc:385] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.541014 20599 raft_consensus.cc:740] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.541208 20597 tablet_bootstrap.cc:492] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.541883 20599 consensus_queue.cc:260] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.548839 20597 tablet_bootstrap.cc:654] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.551429 20599 ts_tablet_manager.cc:1434] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:18:42.552019 20598 tablet_bootstrap.cc:492] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.552695 20598 ts_tablet_manager.cc:1403] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.013s
W20260709 13:18:42.559219 20252 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:42.558292 20598 raft_consensus.cc:359] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.560472 20598 raft_consensus.cc:385] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.560670 20599 tablet_bootstrap.cc:492] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.560843 20598 raft_consensus.cc:740] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.561410 20597 tablet_bootstrap.cc:492] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.561985 20597 ts_tablet_manager.cc:1403] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:18:42.562130 20598 consensus_queue.cc:260] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.564759 20598 ts_tablet_manager.cc:1434] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.007s
I20260709 13:18:42.565146 20597 raft_consensus.cc:359] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.565904 20597 raft_consensus.cc:385] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.566020 20598 tablet_bootstrap.cc:492] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.566237 20597 raft_consensus.cc:740] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.567129 20597 consensus_queue.cc:260] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.568717 20599 tablet_bootstrap.cc:654] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.569748 20597 ts_tablet_manager.cc:1434] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:42.570901 20597 tablet_bootstrap.cc:492] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.573158 20598 tablet_bootstrap.cc:654] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.577716 20597 tablet_bootstrap.cc:654] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.585054 20599 tablet_bootstrap.cc:492] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.585280 20598 tablet_bootstrap.cc:492] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.585716 20599 ts_tablet_manager.cc:1403] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 13:18:42.586055 20598 ts_tablet_manager.cc:1403] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.010s
I20260709 13:18:42.584925 20585 raft_consensus.cc:493] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.587975 20597 tablet_bootstrap.cc:492] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.587762 20585 raft_consensus.cc:515] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.588541 20597 ts_tablet_manager.cc:1403] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.005s
I20260709 13:18:42.589164 20599 raft_consensus.cc:359] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.589707 20598 raft_consensus.cc:359] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.591058 20598 raft_consensus.cc:385] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.591084 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:42.591374 20598 raft_consensus.cc:740] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.591881 20305 raft_consensus.cc:2468] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:42.593142 20418 leader_election.cc:304] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:42.594105 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:42.594684 20597 raft_consensus.cc:359] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.594792 20381 raft_consensus.cc:2468] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:42.595656 20597 raft_consensus.cc:385] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.596081 20597 raft_consensus.cc:740] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.589938 20599 raft_consensus.cc:385] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.597071 20599 raft_consensus.cc:740] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.596920 20597 consensus_queue.cc:260] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.590214 20585 leader_election.cc:290] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:42.598091 20599 consensus_queue.cc:260] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.592226 20598 consensus_queue.cc:260] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.599062 20585 raft_consensus.cc:2804] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.599473 20585 raft_consensus.cc:493] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.599798 20585 raft_consensus.cc:3060] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.600349 20597 ts_tablet_manager.cc:1434] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:18:42.600673 20598 ts_tablet_manager.cc:1434] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.006s
I20260709 13:18:42.601604 20597 tablet_bootstrap.cc:492] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.601608 20598 tablet_bootstrap.cc:492] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.613289 20519 raft_consensus.cc:493] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.613832 20519 raft_consensus.cc:515] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.614784 20598 tablet_bootstrap.cc:654] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.612643 20599 ts_tablet_manager.cc:1434] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:18:42.618556 20519 leader_election.cc:290] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.620142 20597 tablet_bootstrap.cc:654] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.620685 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db17f5622b84012a65f30d5ece1d98c" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:42.621294 20454 raft_consensus.cc:2468] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:42.620251 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db17f5622b84012a65f30d5ece1d98c" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:42.622851 20381 raft_consensus.cc:2468] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:42.622977 20270 leader_election.cc:304] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:42.627939 20519 raft_consensus.cc:2804] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.628677 20519 raft_consensus.cc:493] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.629004 20519 raft_consensus.cc:3060] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.629624 20599 tablet_bootstrap.cc:492] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
W20260709 13:18:42.638032 20418 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.4.65:44339, user_credentials={real_user=slave}} blocked reactor thread for 44978.4us
I20260709 13:18:42.638384 20519 raft_consensus.cc:515] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.639912 20585 raft_consensus.cc:515] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.640558 20519 leader_election.cc:290] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.641710 20613 raft_consensus.cc:493] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.642186 20613 raft_consensus.cc:515] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.642863 20585 leader_election.cc:290] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:42.643092 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:42.643805 20380 raft_consensus.cc:3060] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.643682 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db17f5622b84012a65f30d5ece1d98c" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:42.644326 20613 leader_election.cc:290] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:42.643178 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db17f5622b84012a65f30d5ece1d98c" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:42.645102 20381 raft_consensus.cc:3060] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.645646 20379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
W20260709 13:18:42.647200 20419 leader_election.cc:343] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.644424 20454 raft_consensus.cc:3060] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.644433 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:42.651966 20380 raft_consensus.cc:2468] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:42.652546 20305 raft_consensus.cc:3060] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.653472 20599 tablet_bootstrap.cc:654] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.655277 20381 raft_consensus.cc:2468] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:42.656886 20270 leader_election.cc:304] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:42.657581 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:42.657922 20519 raft_consensus.cc:2804] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:42.659459 20418 leader_election.cc:343] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.659956 20418 leader_election.cc:304] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905; no voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f
I20260709 13:18:42.661293 20614 raft_consensus.cc:2749] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.657721 20419 leader_election.cc:304] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:42.664476 20614 raft_consensus.cc:2804] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.664983 20614 raft_consensus.cc:697] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:42.665807 20614 consensus_queue.cc:237] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.666996 20454 raft_consensus.cc:2468] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:42.673216 20305 raft_consensus.cc:2468] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:42.678442 20598 tablet_bootstrap.cc:492] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.679032 20597 tablet_bootstrap.cc:492] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.679222 20598 ts_tablet_manager.cc:1403] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.001s
I20260709 13:18:42.679879 20519 raft_consensus.cc:697] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:42.681223 20597 ts_tablet_manager.cc:1403] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.080s	user 0.016s	sys 0.016s
I20260709 13:18:42.681048 20519 consensus_queue.cc:237] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.682852 20598 raft_consensus.cc:359] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.683828 20598 raft_consensus.cc:385] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.684182 20598 raft_consensus.cc:740] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.684978 20598 consensus_queue.cc:260] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.687314 20586 raft_consensus.cc:493] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.687618 20598 ts_tablet_manager.cc:1434] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:18:42.688010 20586 raft_consensus.cc:515] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.688736 20598 tablet_bootstrap.cc:492] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.693221 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:42.693938 20454 raft_consensus.cc:2468] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:42.694317 20597 raft_consensus.cc:359] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.695598 20597 raft_consensus.cc:385] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.694093 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:42.692337 20586 leader_election.cc:290] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.697057 20345 leader_election.cc:304] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:42.696170 20597 raft_consensus.cc:740] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.691861 20203 catalog_manager.cc:5697] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:42.704420 20597 consensus_queue.cc:260] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.698885 20305 raft_consensus.cc:2468] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:42.703835 20586 raft_consensus.cc:2804] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.707023 20586 raft_consensus.cc:493] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.707389 20586 raft_consensus.cc:3060] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.708596 20597 ts_tablet_manager.cc:1434] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 13:18:42.710273 20597 tablet_bootstrap.cc:492] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.706140 20599 tablet_bootstrap.cc:492] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.716126 20599 ts_tablet_manager.cc:1403] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.089s	user 0.013s	sys 0.003s
I20260709 13:18:42.715526 20586 raft_consensus.cc:515] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.714419 20201 catalog_manager.cc:5697] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 0 to 1, leader changed from <none> to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 1 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.721426 20586 leader_election.cc:290] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.724349 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:42.725039 20599 raft_consensus.cc:359] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.725741 20599 raft_consensus.cc:385] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.725977 20599 raft_consensus.cc:740] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.726747 20599 consensus_queue.cc:260] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.726987 20598 tablet_bootstrap.cc:654] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.725885 20597 tablet_bootstrap.cc:654] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.729012 20599 ts_tablet_manager.cc:1434] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:18:42.731189 20599 tablet_bootstrap.cc:492] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.736812 20586 raft_consensus.cc:493] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.737483 20586 raft_consensus.cc:515] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.725088 20454 raft_consensus.cc:3060] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.747093 20586 leader_election.cc:290] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.747082 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
W20260709 13:18:42.750301 20344 leader_election.cc:343] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.751777 20453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
W20260709 13:18:42.755131 20345 leader_election.cc:343] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.755710 20345 leader_election.cc:304] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cd696c5af034cc098882be56fff7076; no voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f
I20260709 13:18:42.756594 20586 raft_consensus.cc:2749] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.760107 20454 raft_consensus.cc:2468] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:42.776479 20599 tablet_bootstrap.cc:654] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.777618 20345 leader_election.cc:304] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:42.778601 20586 raft_consensus.cc:2804] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.779130 20586 raft_consensus.cc:697] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:42.780085 20586 consensus_queue.cc:237] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.758435 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:42.785032 20305 raft_consensus.cc:3060] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.785331 20599 tablet_bootstrap.cc:492] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.785928 20599 ts_tablet_manager.cc:1403] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.012s
I20260709 13:18:42.803731 20599 raft_consensus.cc:359] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.804528 20599 raft_consensus.cc:385] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.804872 20599 raft_consensus.cc:740] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.807868 20599 consensus_queue.cc:260] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.815356 20599 ts_tablet_manager.cc:1434] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.007s
I20260709 13:18:42.816220 20586 raft_consensus.cc:493] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.816915 20586 raft_consensus.cc:515] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.818495 20599 tablet_bootstrap.cc:492] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.820207 20304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a258a503dee046afbebc36a2d850f541" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
W20260709 13:18:42.821981 20344 leader_election.cc:343] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.824491 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a258a503dee046afbebc36a2d850f541" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:42.825253 20454 raft_consensus.cc:2468] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:42.827698 20599 tablet_bootstrap.cc:654] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.831095 20345 leader_election.cc:304] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: ec3890a9489c444385a32dde2824973f
I20260709 13:18:42.834582 20572 raft_consensus.cc:2804] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.835021 20572 raft_consensus.cc:493] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.835356 20572 raft_consensus.cc:3060] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.847127 20599 tablet_bootstrap.cc:492] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.847944 20599 ts_tablet_manager.cc:1403] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.017s
I20260709 13:18:42.849004 20586 leader_election.cc:290] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.851485 20599 raft_consensus.cc:359] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.857630 20599 raft_consensus.cc:385] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.858067 20599 raft_consensus.cc:740] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.859004 20599 consensus_queue.cc:260] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.860270 20572 raft_consensus.cc:515] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.863606 20304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a258a503dee046afbebc36a2d850f541" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
W20260709 13:18:42.865150 20344 leader_election.cc:343] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.869966 20599 ts_tablet_manager.cc:1434] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 13:18:42.855997 20305 raft_consensus.cc:2468] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:42.875424 20599 tablet_bootstrap.cc:492] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.886786 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a258a503dee046afbebc36a2d850f541" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:42.887812 20454 raft_consensus.cc:3060] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.889091 20599 tablet_bootstrap.cc:654] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.895717 20572 leader_election.cc:290] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.897408 20454 raft_consensus.cc:2468] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:42.899044 20345 leader_election.cc:304] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: ec3890a9489c444385a32dde2824973f
I20260709 13:18:42.900470 20572 raft_consensus.cc:2804] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.901130 20598 tablet_bootstrap.cc:492] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.901378 20572 raft_consensus.cc:697] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:42.901827 20598 ts_tablet_manager.cc:1403] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.213s	user 0.077s	sys 0.086s
I20260709 13:18:42.902189 20597 tablet_bootstrap.cc:492] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.902717 20572 consensus_queue.cc:237] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.904028 20599 tablet_bootstrap.cc:492] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.904649 20599 ts_tablet_manager.cc:1403] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.001s
I20260709 13:18:42.905093 20598 raft_consensus.cc:359] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.905870 20598 raft_consensus.cc:385] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.905448 20204 catalog_manager.cc:5697] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.906160 20598 raft_consensus.cc:740] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.907074 20598 consensus_queue.cc:260] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.907763 20599 raft_consensus.cc:359] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.908519 20599 raft_consensus.cc:385] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.908818 20599 raft_consensus.cc:740] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.909631 20598 ts_tablet_manager.cc:1434] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:42.909744 20599 consensus_queue.cc:260] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.910713 20598 tablet_bootstrap.cc:492] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.912202 20599 ts_tablet_manager.cc:1434] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:42.913210 20599 tablet_bootstrap.cc:492] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.917690 20598 tablet_bootstrap.cc:654] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.919345 20599 tablet_bootstrap.cc:654] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.902701 20597 ts_tablet_manager.cc:1403] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.193s	user 0.076s	sys 0.074s
I20260709 13:18:42.926379 20597 raft_consensus.cc:359] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.927150 20597 raft_consensus.cc:385] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.927706 20597 raft_consensus.cc:740] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.928597 20597 consensus_queue.cc:260] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.931470 20597 ts_tablet_manager.cc:1434] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:18:42.932569 20597 tablet_bootstrap.cc:492] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.945667 20597 tablet_bootstrap.cc:654] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.948079 20201 catalog_manager.cc:5697] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.950700 20599 tablet_bootstrap.cc:492] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:42.950902 20598 tablet_bootstrap.cc:492] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:42.953714 20598 ts_tablet_manager.cc:1403] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.006s
I20260709 13:18:42.953714 20599 ts_tablet_manager.cc:1403] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.012s
I20260709 13:18:42.956328 20597 tablet_bootstrap.cc:492] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:42.956348 20598 raft_consensus.cc:359] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.957024 20597 ts_tablet_manager.cc:1403] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.004s
I20260709 13:18:42.957227 20598 raft_consensus.cc:385] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.957507 20598 raft_consensus.cc:740] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.958199 20599 raft_consensus.cc:359] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.958302 20598 consensus_queue.cc:260] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.958981 20599 raft_consensus.cc:385] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.959301 20599 raft_consensus.cc:740] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.960052 20599 consensus_queue.cc:260] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.960142 20597 raft_consensus.cc:359] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.960817 20597 raft_consensus.cc:385] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.961069 20597 raft_consensus.cc:740] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.961979 20599 ts_tablet_manager.cc:1434] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:42.961834 20597 consensus_queue.cc:260] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.963593 20599 tablet_bootstrap.cc:492] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f: Bootstrap starting.
I20260709 13:18:42.964026 20597 ts_tablet_manager.cc:1434] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:42.966161 20597 tablet_bootstrap.cc:492] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076: Bootstrap starting.
I20260709 13:18:42.969048 20572 raft_consensus.cc:493] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.969635 20572 raft_consensus.cc:515] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:42.965013 20598 ts_tablet_manager.cc:1434] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:18:42.972049 20599 tablet_bootstrap.cc:654] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.972157 20598 tablet_bootstrap.cc:492] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905: Bootstrap starting.
I20260709 13:18:42.972986 20597 tablet_bootstrap.cc:654] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.974224 20572 leader_election.cc:290] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:42.976073 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e773ebd3ba0400181d5d44339c22cd1" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:42.976832 20454 raft_consensus.cc:2468] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:42.978204 20345 leader_election.cc:304] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:42.980227 20626 raft_consensus.cc:2804] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.980597 20626 raft_consensus.cc:493] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.980921 20626 raft_consensus.cc:3060] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.981377 20598 tablet_bootstrap.cc:654] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.972878 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e773ebd3ba0400181d5d44339c22cd1" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:42.988339 20305 raft_consensus.cc:2468] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:43.003712 20597 tablet_bootstrap.cc:492] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076: No bootstrap required, opened a new log
I20260709 13:18:43.004290 20597 ts_tablet_manager.cc:1403] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.011s
I20260709 13:18:43.006289 20597 raft_consensus.cc:359] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.006800 20597 raft_consensus.cc:385] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.007005 20597 raft_consensus.cc:740] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.007579 20597 consensus_queue.cc:260] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.009747 20597 ts_tablet_manager.cc:1434] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:43.019382 20626 raft_consensus.cc:515] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.022439 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e773ebd3ba0400181d5d44339c22cd1" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:43.023068 20305 raft_consensus.cc:3060] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.023000 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e773ebd3ba0400181d5d44339c22cd1" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:43.024009 20454 raft_consensus.cc:3060] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.026273 20598 tablet_bootstrap.cc:492] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905: No bootstrap required, opened a new log
I20260709 13:18:43.026986 20598 ts_tablet_manager.cc:1403] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.005s
I20260709 13:18:43.023211 20626 leader_election.cc:290] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.030704 20598 raft_consensus.cc:359] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.031011 20305 raft_consensus.cc:2468] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:43.031244 20598 raft_consensus.cc:385] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.031658 20598 raft_consensus.cc:740] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.032325 20344 leader_election.cc:304] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.032609 20614 raft_consensus.cc:493] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.033311 20626 raft_consensus.cc:2804] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.035367 20618 raft_consensus.cc:493] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.033021 20599 tablet_bootstrap.cc:492] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f: No bootstrap required, opened a new log
I20260709 13:18:43.033159 20614 raft_consensus.cc:515] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.036341 20599 ts_tablet_manager.cc:1403] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f: Time spent bootstrapping tablet: real 0.073s	user 0.024s	sys 0.010s
I20260709 13:18:43.036142 20618 raft_consensus.cc:515] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.036975 20626 raft_consensus.cc:697] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:43.037958 20626 consensus_queue.cc:237] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.038725 20614 leader_election.cc:290] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:43.039012 20381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:43.039834 20381 raft_consensus.cc:2468] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:43.038372 20618 leader_election.cc:290] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.040274 20453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:43.040343 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:43.041410 20380 raft_consensus.cc:2468] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:43.041780 20419 leader_election.cc:304] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:43.042039 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:43.032397 20454 raft_consensus.cc:2468] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:43.042812 20270 leader_election.cc:304] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.043855 20618 raft_consensus.cc:2804] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.044235 20618 raft_consensus.cc:493] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.032366 20598 consensus_queue.cc:260] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.044584 20599 raft_consensus.cc:359] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.045445 20599 raft_consensus.cc:385] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.045463 20613 raft_consensus.cc:2804] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.041152 20453 raft_consensus.cc:2468] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 0.
I20260709 13:18:43.044615 20618 raft_consensus.cc:3060] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.045828 20599 raft_consensus.cc:740] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.107312 20599 consensus_queue.cc:260] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.107926 20598 ts_tablet_manager.cc:1434] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905: Time spent starting tablet: real 0.081s	user 0.005s	sys 0.004s
I20260709 13:18:43.112491 20599 ts_tablet_manager.cc:1434] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f: Time spent starting tablet: real 0.076s	user 0.005s	sys 0.025s
I20260709 13:18:43.112619 20204 catalog_manager.cc:5697] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.046056 20305 raft_consensus.cc:2468] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:43.116082 20626 raft_consensus.cc:493] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.116688 20626 raft_consensus.cc:515] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.118770 20626 leader_election.cc:290] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.108971 20613 raft_consensus.cc:493] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.124732 20613 raft_consensus.cc:3060] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.125025 20453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:43.125279 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:43.125851 20453 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:43.126088 20305 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 0.
I20260709 13:18:43.127203 20345 leader_election.cc:304] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:43.128080 20572 raft_consensus.cc:2804] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.128436 20572 raft_consensus.cc:493] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.128764 20572 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.134176 20614 consensus_queue.cc:1048] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.134420 20613 raft_consensus.cc:515] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.136762 20613 leader_election.cc:290] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:43.136639 20618 raft_consensus.cc:515] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.138571 20618 leader_election.cc:290] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.139098 20618 raft_consensus.cc:493] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.139523 20618 raft_consensus.cc:515] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.140125 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:43.140770 20380 raft_consensus.cc:3060] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.141538 20618 leader_election.cc:290] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.145869 20572 raft_consensus.cc:515] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.148132 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:43.148769 20304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:43.149765 20305 raft_consensus.cc:3060] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.150378 20379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:43.150979 20379 raft_consensus.cc:3060] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.149765 20304 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.156360 20378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a258a503dee046afbebc36a2d850f541" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:43.157442 20305 raft_consensus.cc:2468] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:43.159278 20418 leader_election.cc:304] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.159761 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a258a503dee046afbebc36a2d850f541" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905" is_pre_election: true
I20260709 13:18:43.160542 20454 raft_consensus.cc:2393] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec3890a9489c444385a32dde2824973f in current term 1: Already voted for candidate 4cd696c5af034cc098882be56fff7076 in this term.
I20260709 13:18:43.160622 20585 raft_consensus.cc:2804] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.161141 20585 raft_consensus.cc:697] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:43.161950 20585 consensus_queue.cc:237] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.162307 20270 leader_election.cc:304] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec3890a9489c444385a32dde2824973f; no voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076
I20260709 13:18:43.163092 20618 raft_consensus.cc:3060] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.163601 20614 consensus_queue.cc:1048] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.164526 20453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:43.165113 20453 raft_consensus.cc:3060] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.174469 20618 raft_consensus.cc:2749] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:43.178468 20379 raft_consensus.cc:2468] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:43.180482 20380 raft_consensus.cc:2468] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:43.184540 20572 leader_election.cc:290] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.184675 20270 leader_election.cc:304] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.185670 20618 raft_consensus.cc:2804] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.185757 20453 raft_consensus.cc:2468] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 1.
I20260709 13:18:43.186174 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:43.186205 20618 raft_consensus.cc:697] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:43.186896 20454 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.188555 20618 consensus_queue.cc:237] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.193755 20454 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:43.194958 20345 leader_election.cc:304] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:43.195902 20572 raft_consensus.cc:2804] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.197345 20572 raft_consensus.cc:697] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:43.182421 20304 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 1.
I20260709 13:18:43.182024 20204 catalog_manager.cc:5697] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.198102 20572 consensus_queue.cc:237] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.210889 20204 catalog_manager.cc:5697] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.211953 20201 catalog_manager.cc:5697] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 0 to 1, leader changed from <none> to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 1 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.217132 20618 consensus_queue.cc:1048] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.227804 20639 raft_consensus.cc:493] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.229315 20639 raft_consensus.cc:515] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.229997 20618 consensus_queue.cc:1048] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.233157 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a144f8a86ec4b9982e6c06ac872b1fd" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f" is_pre_election: true
I20260709 13:18:43.233786 20306 raft_consensus.cc:2468] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:43.235057 20418 leader_election.cc:304] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.236078 20613 raft_consensus.cc:2804] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.236461 20613 raft_consensus.cc:493] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.236796 20613 raft_consensus.cc:3060] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.237303 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a144f8a86ec4b9982e6c06ac872b1fd" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076" is_pre_election: true
I20260709 13:18:43.237985 20380 raft_consensus.cc:2468] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 0.
I20260709 13:18:43.242270 20639 leader_election.cc:290] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:43.255256 20613 raft_consensus.cc:515] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.257562 20613 leader_election.cc:290] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:43.258414 20304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a144f8a86ec4b9982e6c06ac872b1fd" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:43.258906 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a144f8a86ec4b9982e6c06ac872b1fd" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:43.259081 20304 raft_consensus.cc:3060] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.259509 20380 raft_consensus.cc:3060] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.265537 20380 raft_consensus.cc:2468] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:43.266443 20419 leader_election.cc:304] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:43.266952 20304 raft_consensus.cc:2468] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 1.
I20260709 13:18:43.267074 20613 raft_consensus.cc:2804] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.267669 20613 raft_consensus.cc:697] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:43.268359 20613 consensus_queue.cc:237] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.274869 20201 catalog_manager.cc:5697] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 1 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.297094 20454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
mode: GRACEFUL
new_leader_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:44544
I20260709 13:18:43.297606 20454 raft_consensus.cc:606] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Received request to transfer leadership to TS ec3890a9489c444385a32dde2824973f
I20260709 13:18:43.302053 20454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
mode: GRACEFUL
new_leader_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:44544
I20260709 13:18:43.302534 20454 raft_consensus.cc:606] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Received request to transfer leadership to TS ec3890a9489c444385a32dde2824973f
I20260709 13:18:43.309029 20380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
mode: GRACEFUL
new_leader_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:48436
I20260709 13:18:43.309479 20380 raft_consensus.cc:606] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Received request to transfer leadership to TS ec3890a9489c444385a32dde2824973f
I20260709 13:18:43.314142 20304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
mode: GRACEFUL
new_leader_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:37288
I20260709 13:18:43.314749 20304 raft_consensus.cc:606] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Received request to transfer leadership to TS 4cd696c5af034cc098882be56fff7076
I20260709 13:18:43.350170 20572 consensus_queue.cc:1048] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.360567 20586 consensus_queue.cc:1048] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.361111 20572 consensus_queue.cc:1048] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.377885 20572 consensus_queue.cc:1048] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.506359 20572 consensus_queue.cc:1048] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.527611 20652 consensus_queue.cc:1048] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.552381 20613 consensus_queue.cc:1048] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.565506 20585 consensus_queue.cc:1048] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.579916 20639 raft_consensus.cc:993] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: : Instructing follower ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:43.580389 20585 raft_consensus.cc:1081] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Signalling peer ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:43.582187 20306 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:49052
I20260709 13:18:43.582907 20306 raft_consensus.cc:493] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:43.583323 20306 raft_consensus.cc:3060] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.591995 20306 raft_consensus.cc:515] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.594211 20306 leader_election.cc:290] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.595564 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:43.596225 20380 raft_consensus.cc:3060] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.595654 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfc7b968a2a42dc92565aa8335ea320" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:43.597070 20454 raft_consensus.cc:3055] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:43.597345 20454 raft_consensus.cc:740] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:43.598160 20454 consensus_queue.cc:260] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.599109 20454 raft_consensus.cc:3060] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.602818 20380 raft_consensus.cc:2468] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:43.604089 20270 leader_election.cc:304] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.604948 20519 raft_consensus.cc:2804] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:43.605324 20519 raft_consensus.cc:697] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:43.605497 20454 raft_consensus.cc:2468] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:43.606377 20519 consensus_queue.cc:237] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.612901 20201 catalog_manager.cc:5697] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 1 to 2, leader changed from 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67) to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 2 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.665946 20639 consensus_queue.cc:1048] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.677600 20613 consensus_queue.cc:1048] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.684557 20618 consensus_queue.cc:1048] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.698966 20519 consensus_queue.cc:1048] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.703624 20572 consensus_queue.cc:1048] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.709688 20618 raft_consensus.cc:993] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: : Instructing follower 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:43.710160 20618 raft_consensus.cc:1081] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Signalling peer 4cd696c5af034cc098882be56fff7076 to start an election
I20260709 13:18:43.712289 20380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
 from {username='slave'} at 127.0.0.1:51678
I20260709 13:18:43.712836 20380 raft_consensus.cc:493] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:43.713212 20380 raft_consensus.cc:3060] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.721784 20380 raft_consensus.cc:515] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.724848 20306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:43.725471 20306 raft_consensus.cc:3055] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:43.725824 20306 raft_consensus.cc:740] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:43.726769 20306 consensus_queue.cc:260] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.728561 20380 leader_election.cc:290] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.729723 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8717bb64ae04786857242f5653cc8ee" candidate_uuid: "4cd696c5af034cc098882be56fff7076" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:43.730304 20454 raft_consensus.cc:3060] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.730803 20572 consensus_queue.cc:1048] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.731262 20306 raft_consensus.cc:3060] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.738610 20454 raft_consensus.cc:2468] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
I20260709 13:18:43.739962 20306 raft_consensus.cc:2468] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd696c5af034cc098882be56fff7076 in term 2.
I20260709 13:18:43.740110 20345 leader_election.cc:304] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:43.741405 20572 raft_consensus.cc:2804] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:43.741911 20572 raft_consensus.cc:697] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Becoming Leader. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:43.742928 20572 consensus_queue.cc:237] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.749845 20585 raft_consensus.cc:993] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: : Instructing follower ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:43.750401 20585 raft_consensus.cc:1081] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Signalling peer ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:43.752492 20305 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:49052
I20260709 13:18:43.753118 20305 raft_consensus.cc:493] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:43.753504 20305 raft_consensus.cc:3060] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.754626 20586 raft_consensus.cc:993] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: : Instructing follower ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:43.755093 20572 raft_consensus.cc:1081] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Signalling peer ec3890a9489c444385a32dde2824973f to start an election
I20260709 13:18:43.756723 20304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
 from {username='slave'} at 127.0.0.1:49060
I20260709 13:18:43.757302 20304 raft_consensus.cc:493] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:43.757759 20304 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.757103 20203 catalog_manager.cc:5697] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 reported cstate change: term changed from 1 to 2, leader changed from ec3890a9489c444385a32dde2824973f (127.10.4.65) to 4cd696c5af034cc098882be56fff7076 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "4cd696c5af034cc098882be56fff7076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.761375 20305 raft_consensus.cc:515] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.764106 20380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:43.764644 20380 raft_consensus.cc:3060] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.764730 20305 leader_election.cc:290] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.765061 20304 raft_consensus.cc:515] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.764586 20454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f281ecea50c24327b8006c8f08c72736" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:43.766261 20454 raft_consensus.cc:3055] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:43.766559 20454 raft_consensus.cc:740] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:43.767613 20454 consensus_queue.cc:260] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.768935 20454 raft_consensus.cc:3060] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.770673 20379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:43.771304 20379 raft_consensus.cc:3055] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:43.771641 20379 raft_consensus.cc:740] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:43.772223 20380 raft_consensus.cc:2468] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:43.772611 20379 consensus_queue.cc:260] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.773497 20304 leader_election.cc:290] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Requested vote from peers 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101), 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67:33527)
I20260709 13:18:43.773824 20379 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.774693 20453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "ec3890a9489c444385a32dde2824973f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
I20260709 13:18:43.775198 20454 raft_consensus.cc:2468] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:43.775275 20453 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:43.776062 20270 leader_election.cc:304] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.777174 20519 raft_consensus.cc:2804] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:43.777642 20519 raft_consensus.cc:697] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:43.778633 20519 consensus_queue.cc:237] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.781510 20379 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:43.783713 20453 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec3890a9489c444385a32dde2824973f in term 2.
I20260709 13:18:43.785131 20270 leader_election.cc:304] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd696c5af034cc098882be56fff7076, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:43.786509 20519 raft_consensus.cc:2804] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:43.786980 20519 raft_consensus.cc:697] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Leader. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:43.787431 20204 catalog_manager.cc:5697] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 1 to 2, leader changed from 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67) to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 2 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.787901 20519 consensus_queue.cc:237] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:43.801024 20204 catalog_manager.cc:5697] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f reported cstate change: term changed from 1 to 2, leader changed from 4cd696c5af034cc098882be56fff7076 (127.10.4.66) to ec3890a9489c444385a32dde2824973f (127.10.4.65). New cstate: current_term: 2 leader_uuid: "ec3890a9489c444385a32dde2824973f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.170995 20453 raft_consensus.cc:1275] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.172495 20519 consensus_queue.cc:1048] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:44.178287 20453 raft_consensus.cc:1275] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.181447 20379 raft_consensus.cc:1275] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.183355 20646 consensus_queue.cc:1048] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:44.182775 20519 consensus_queue.cc:1048] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:44.188676 20380 raft_consensus.cc:1275] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.190234 20646 consensus_queue.cc:1048] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:44.198369 20379 raft_consensus.cc:1275] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.200058 20646 consensus_queue.cc:1048] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:44.204998 20453 raft_consensus.cc:1275] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer ec3890a9489c444385a32dde2824973f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.211721 20646 consensus_queue.cc:1048] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:44.247092 20304 raft_consensus.cc:1275] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.248332 20572 consensus_queue.cc:1048] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:44.256837 20453 raft_consensus.cc:1275] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Refusing update from remote peer 4cd696c5af034cc098882be56fff7076: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:44.260732 20652 consensus_queue.cc:1048] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:46.327705 20204 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59046:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:18:46.348986 20204 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59046:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:18:46.375240 10257 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:18:46.498159 20698 tablet.cc:1724] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.504099 20699 tablet.cc:1724] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:46.524008 20481 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:c8717bb64ae04786857242f5653cc8ee)
W20260709 13:18:46.540701 20481 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:c8717bb64ae04786857242f5653cc8ee)
W20260709 13:18:46.542979 20481 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:c8717bb64ae04786857242f5653cc8ee)
W20260709 13:18:46.573676 20694 log.cc:927] Time spent T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 13:18:46.639648 20693 log.cc:927] Time spent T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:18:46.641324 20703 log.cc:927] Time spent T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:46.660326 20304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
mode: GRACEFUL
new_leader_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:37298
I20260709 13:18:46.670991 20304 raft_consensus.cc:606] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Received request to transfer leadership to TS 4c7e3a8fef9c47b5b879adb450333905
W20260709 13:18:46.684162 20345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:33527, user_credentials={real_user=slave}} blocked reactor thread for 133729us
I20260709 13:18:46.749095 20699 tablet.cc:1724] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:46.761582 20344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.65:44339, user_credentials={real_user=slave}} blocked reactor thread for 68822.4us
I20260709 13:18:46.669904 20724 tablet.cc:1724] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.738385 20698 tablet.cc:1724] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.803791 20737 tablet.cc:1724] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:46.804163 20345 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:33527, user_credentials={real_user=slave}} blocked reactor thread for 80806.6us
I20260709 13:18:46.809401 20721 tablet.cc:1724] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.810482 20727 tablet.cc:1724] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.783571 20732 tablet.cc:1724] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.844954 20736 tablet.cc:1724] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.847266 20716 raft_consensus.cc:993] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: : Instructing follower 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:46.814113 20699 tablet.cc:1724] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.869019 20669 raft_consensus.cc:1081] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Signalling peer 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:46.869777 20743 tablet.cc:1724] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:46.866696 20733 log.cc:927] Time spent T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
I20260709 13:18:46.849298 20725 tablet.cc:1724] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.801941 20738 tablet.cc:1724] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.872884 20451 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:44512
I20260709 13:18:46.875694 20451 raft_consensus.cc:493] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:46.876088 20451 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:46.884501 20740 tablet.cc:1724] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.886886 20737 tablet.cc:1724] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.894547 20698 tablet.cc:1724] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.909996 20699 tablet.cc:1724] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.916034 20732 tablet.cc:1724] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.910552 20451 raft_consensus.cc:515] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:46.929579 20451 leader_election.cc:290] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 3 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:46.932547 20379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:46.933251 20379 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:46.939864 20698 tablet.cc:1724] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.920866 20740 tablet.cc:1724] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.941442 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "964f1f17376d48538f5f1827dba39ee5" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:46.943574 20305 raft_consensus.cc:3055] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:46.943948 20305 raft_consensus.cc:740] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:46.944836 20305 consensus_queue.cc:260] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:46.938055 20727 tablet.cc:1724] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.948033 20305 raft_consensus.cc:3060] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:46.950196 20738 tablet.cc:1724] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.954555 20724 tablet.cc:1724] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.957507 20743 tablet.cc:1724] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.966408 20737 tablet.cc:1724] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.974831 20379 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 3.
I20260709 13:18:46.978983 20305 raft_consensus.cc:2468] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 3.
I20260709 13:18:46.979477 20419 leader_election.cc:304] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:46.924860 20699 tablet.cc:1724] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:46.982473 20614 raft_consensus.cc:2804] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:46.983011 20614 raft_consensus.cc:697] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:46.984170 20614 consensus_queue.cc:237] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.001746 20379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
mode: GRACEFUL
new_leader_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:48440
I20260709 13:18:47.002241 20379 raft_consensus.cc:606] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Received request to transfer leadership to TS 4c7e3a8fef9c47b5b879adb450333905
I20260709 13:18:47.003638 10257 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:18:47.004515 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:47.005967 20202 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa: soft_deleted_table [id=32911d2361cd41cfb98c63b43cdea20a] alter complete (version 1)
I20260709 13:18:47.014705 20379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e"
dest_uuid: "4cd696c5af034cc098882be56fff7076"
mode: GRACEFUL
new_leader_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:48440
I20260709 13:18:47.015301 20379 raft_consensus.cc:606] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Received request to transfer leadership to TS 4c7e3a8fef9c47b5b879adb450333905
I20260709 13:18:47.016906 20202 catalog_manager.cc:5697] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 2 to 3, leader changed from ec3890a9489c444385a32dde2824973f (127.10.4.65) to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.017827 20305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6"
dest_uuid: "ec3890a9489c444385a32dde2824973f"
mode: GRACEFUL
new_leader_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:37298
I20260709 13:18:47.018291 20305 raft_consensus.cc:606] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Received request to transfer leadership to TS 4c7e3a8fef9c47b5b879adb450333905
I20260709 13:18:47.019438 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:47.019856 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:47.125411 20715 raft_consensus.cc:993] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: : Instructing follower 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:47.126022 20739 raft_consensus.cc:1081] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Signalling peer 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:47.127486 20454 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:44518
I20260709 13:18:47.127955 20454 raft_consensus.cc:493] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:47.128279 20454 raft_consensus.cc:3060] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.133216 20454 raft_consensus.cc:515] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.134578 20454 leader_election.cc:290] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 2 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:47.135389 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:47.135638 20379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dc85cb531340ffa57355c8424590bc" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:47.135951 20305 raft_consensus.cc:3060] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.136232 20379 raft_consensus.cc:3055] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:47.136485 20379 raft_consensus.cc:740] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:47.137111 20379 consensus_queue.cc:260] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.138049 20379 raft_consensus.cc:3060] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.141886 20305 raft_consensus.cc:2468] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 2.
I20260709 13:18:47.143028 20418 leader_election.cc:304] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:47.143263 20379 raft_consensus.cc:2468] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 2.
I20260709 13:18:47.143780 20614 raft_consensus.cc:2804] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:47.144531 20614 raft_consensus.cc:697] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 2 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:47.145241 20614 consensus_queue.cc:237] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.151829 20203 catalog_manager.cc:5697] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 1 to 2, leader changed from 4cd696c5af034cc098882be56fff7076 (127.10.4.66) to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.231617 20715 raft_consensus.cc:993] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076: : Instructing follower 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:47.232016 20715 raft_consensus.cc:1081] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Signalling peer 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:47.233350 20454 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:44518
I20260709 13:18:47.233795 20454 raft_consensus.cc:493] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:47.234054 20454 raft_consensus.cc:3060] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.238498 20454 raft_consensus.cc:515] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.239907 20454 leader_election.cc:290] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 2 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:47.240583 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:47.240935 20379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a87f179730e34788b5ebbcebdd11741e" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:47.241127 20305 raft_consensus.cc:3060] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.241459 20379 raft_consensus.cc:3055] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:47.241727 20379 raft_consensus.cc:740] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4cd696c5af034cc098882be56fff7076, State: Running, Role: LEADER
I20260709 13:18:47.242430 20379 consensus_queue.cc:260] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.243345 20379 raft_consensus.cc:3060] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:47.245482 20305 raft_consensus.cc:2468] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 2.
I20260709 13:18:47.246379 20418 leader_election.cc:304] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, ec3890a9489c444385a32dde2824973f; no voters: 
I20260709 13:18:47.247046 20614 raft_consensus.cc:2804] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:47.247534 20614 raft_consensus.cc:697] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:47.248472 20614 consensus_queue.cc:237] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.248776 20379 raft_consensus.cc:2468] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 2.
I20260709 13:18:47.256407 20203 catalog_manager.cc:5697] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 1 to 2, leader changed from 4cd696c5af034cc098882be56fff7076 (127.10.4.66) to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 2 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.268669 20716 raft_consensus.cc:993] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f: : Instructing follower 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:47.269187 20711 raft_consensus.cc:1081] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Signalling peer 4c7e3a8fef9c47b5b879adb450333905 to start an election
I20260709 13:18:47.270388 20454 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6"
dest_uuid: "4c7e3a8fef9c47b5b879adb450333905"
 from {username='slave'} at 127.0.0.1:44512
I20260709 13:18:47.270804 20454 raft_consensus.cc:493] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:47.271052 20454 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:47.276090 20454 raft_consensus.cc:515] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.277451 20454 leader_election.cc:290] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 3 election: Requested vote from peers ec3890a9489c444385a32dde2824973f (127.10.4.65:44339), 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101)
I20260709 13:18:47.278147 20305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ec3890a9489c444385a32dde2824973f"
I20260709 13:18:47.278759 20305 raft_consensus.cc:3055] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:47.278637 20377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14b49abcadf494fb27aca35637c2bd6" candidate_uuid: "4c7e3a8fef9c47b5b879adb450333905" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4cd696c5af034cc098882be56fff7076"
I20260709 13:18:47.279139 20305 raft_consensus.cc:740] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ec3890a9489c444385a32dde2824973f, State: Running, Role: LEADER
I20260709 13:18:47.279345 20377 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:47.279953 20305 consensus_queue.cc:260] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.281260 20305 raft_consensus.cc:3060] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:47.286633 20377 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 3.
I20260709 13:18:47.287547 20305 raft_consensus.cc:2468] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c7e3a8fef9c47b5b879adb450333905 in term 3.
I20260709 13:18:47.287871 20419 leader_election.cc:304] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7e3a8fef9c47b5b879adb450333905, 4cd696c5af034cc098882be56fff7076; no voters: 
I20260709 13:18:47.288714 20614 raft_consensus.cc:2804] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:47.289139 20614 raft_consensus.cc:697] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 LEADER]: Becoming Leader. State: Replica: 4c7e3a8fef9c47b5b879adb450333905, State: Running, Role: LEADER
I20260709 13:18:47.289920 20614 consensus_queue.cc:237] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } }
I20260709 13:18:47.296442 20203 catalog_manager.cc:5697] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 reported cstate change: term changed from 2 to 3, leader changed from ec3890a9489c444385a32dde2824973f (127.10.4.65) to 4c7e3a8fef9c47b5b879adb450333905 (127.10.4.67). New cstate: current_term: 3 leader_uuid: "4c7e3a8fef9c47b5b879adb450333905" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7e3a8fef9c47b5b879adb450333905" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.521631 20305 raft_consensus.cc:1275] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:47.522872 20614 consensus_queue.cc:1048] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:47.530745 20377 raft_consensus.cc:1275] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:47.532274 20710 consensus_queue.cc:1048] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:47.675031 20305 raft_consensus.cc:1275] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:47.676561 20614 consensus_queue.cc:1048] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:47.682487 20377 raft_consensus.cc:1275] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:47.684130 20614 consensus_queue.cc:1048] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:47.686704 20377 raft_consensus.cc:1275] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:47.688251 20751 consensus_queue.cc:1048] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:47.703274 20305 raft_consensus.cc:1275] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:47.707115 20751 consensus_queue.cc:1048] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:47.727027 20377 raft_consensus.cc:1275] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:47.728195 20751 consensus_queue.cc:1048] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd696c5af034cc098882be56fff7076" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 39101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:47.735899 20305 raft_consensus.cc:1275] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Refusing update from remote peer 4c7e3a8fef9c47b5b879adb450333905: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:18:47.737360 20710 consensus_queue.cc:1048] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec3890a9489c444385a32dde2824973f" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:48.073799 10257 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:18:48.078962 10257 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:18:48.079428 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:48.084342 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:18:48.084812 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:49.114275 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
W20260709 13:18:49.127743 20418 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 13:18:49.131923 20418 consensus_peers.cc:597] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 -> Peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Couldn't send request to peer ec3890a9489c444385a32dde2824973f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.137995 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:49.138943 10257 tablet_replica.cc:333] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.139971 10257 raft_consensus.cc:2243] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.140547 10257 raft_consensus.cc:2272] T 4a144f8a86ec4b9982e6c06ac872b1fd P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:49.141180 20418 consensus_peers.cc:597] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 -> Peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Couldn't send request to peer ec3890a9489c444385a32dde2824973f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.142823 10257 tablet_replica.cc:333] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.143589 10257 raft_consensus.cc:2243] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.144009 10257 raft_consensus.cc:2272] T a258a503dee046afbebc36a2d850f541 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.145888 10257 tablet_replica.cc:333] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.146404 10257 raft_consensus.cc:2243] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.146804 10257 raft_consensus.cc:2272] T a5dc85cb531340ffa57355c8424590bc P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.148728 10257 tablet_replica.cc:333] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.149209 10257 raft_consensus.cc:2243] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.150089 10257 raft_consensus.cc:2272] T 8db17f5622b84012a65f30d5ece1d98c P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.152398 10257 tablet_replica.cc:333] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.153075 10257 raft_consensus.cc:2243] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.153494 10257 raft_consensus.cc:2272] T ab534676680e4b1184abab27ddb6752c P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.155616 10257 tablet_replica.cc:333] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.156368 10257 raft_consensus.cc:2243] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.156855 10257 raft_consensus.cc:2272] T c7bb9c20fc774e248068eb0fb922884d P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.158953 10257 tablet_replica.cc:333] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.159586 10257 raft_consensus.cc:2243] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.159992 10257 raft_consensus.cc:2272] T e14b49abcadf494fb27aca35637c2bd6 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.162235 10257 tablet_replica.cc:333] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.162815 10257 raft_consensus.cc:2243] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.163276 10257 raft_consensus.cc:2272] T a87f179730e34788b5ebbcebdd11741e P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:49.165729 20418 consensus_peers.cc:597] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 -> Peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Couldn't send request to peer ec3890a9489c444385a32dde2824973f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.165915 10257 tablet_replica.cc:333] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.166692 10257 raft_consensus.cc:2243] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.167436 10257 raft_consensus.cc:2272] T 0dfc7b968a2a42dc92565aa8335ea320 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.169612 10257 tablet_replica.cc:333] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.170367 10257 raft_consensus.cc:2243] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.170822 10257 raft_consensus.cc:2272] T 0cd187256e1c4ee3a19a1f4ee62d3dea P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.173105 10257 tablet_replica.cc:333] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.173664 10257 raft_consensus.cc:2243] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.174081 10257 raft_consensus.cc:2272] T 964f1f17376d48538f5f1827dba39ee5 P ec3890a9489c444385a32dde2824973f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.175978 10257 tablet_replica.cc:333] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.176466 10257 raft_consensus.cc:2243] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.176867 10257 raft_consensus.cc:2272] T 259855087e02457cb10e70f2808110b8 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.178933 10257 tablet_replica.cc:333] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.179486 10257 raft_consensus.cc:2243] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.180359 10257 raft_consensus.cc:2272] T c0427a1799c24e9cb0546a29f73b1714 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.182514 10257 tablet_replica.cc:333] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.183209 10257 raft_consensus.cc:2243] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.184122 10257 raft_consensus.cc:2272] T f281ecea50c24327b8006c8f08c72736 P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.186112 10257 tablet_replica.cc:333] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.186653 10257 raft_consensus.cc:2243] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.187425 10257 raft_consensus.cc:2272] T b6ac007579544fe196e9a79834912e9e P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.189456 10257 tablet_replica.cc:333] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.189915 10257 raft_consensus.cc:2243] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.190286 10257 raft_consensus.cc:2272] T 2e773ebd3ba0400181d5d44339c22cd1 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.192173 10257 tablet_replica.cc:333] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.192610 10257 raft_consensus.cc:2243] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.192966 10257 raft_consensus.cc:2272] T c8717bb64ae04786857242f5653cc8ee P ec3890a9489c444385a32dde2824973f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.194907 10257 tablet_replica.cc:333] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f: stopping tablet replica
I20260709 13:18:49.195348 10257 raft_consensus.cc:2243] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.196131 10257 raft_consensus.cc:2272] T b2b5471415d545cd8e6fc1c4f2711a88 P ec3890a9489c444385a32dde2824973f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:49.208789 20418 consensus_peers.cc:597] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 -> Peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Couldn't send request to peer ec3890a9489c444385a32dde2824973f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:49.229709 20344 consensus_peers.cc:597] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 -> Peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Couldn't send request to peer ec3890a9489c444385a32dde2824973f. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:44339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:49.231510 20344 consensus_peers.cc:597] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 -> Peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Couldn't send request to peer ec3890a9489c444385a32dde2824973f. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:44339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.237146 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
I20260709 13:18:49.274492 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
I20260709 13:18:49.299256 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:49.300055 10257 tablet_replica.cc:333] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.300971 10257 raft_consensus.cc:2243] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.301594 10257 raft_consensus.cc:2272] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.303823 10257 tablet_replica.cc:333] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.304414 10257 raft_consensus.cc:2243] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.305166 10257 raft_consensus.cc:2272] T a258a503dee046afbebc36a2d850f541 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.307435 10257 tablet_replica.cc:333] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.307981 10257 raft_consensus.cc:2243] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.308372 10257 raft_consensus.cc:2272] T c7bb9c20fc774e248068eb0fb922884d P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.310449 10257 tablet_replica.cc:333] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.310998 10257 raft_consensus.cc:2243] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.311780 10257 raft_consensus.cc:2272] T c8717bb64ae04786857242f5653cc8ee P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.313705 10257 tablet_replica.cc:333] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.314239 10257 raft_consensus.cc:2243] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.314975 10257 raft_consensus.cc:2272] T 2e773ebd3ba0400181d5d44339c22cd1 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.316883 10257 tablet_replica.cc:333] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.317457 10257 raft_consensus.cc:2243] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.317843 10257 raft_consensus.cc:2272] T b2b5471415d545cd8e6fc1c4f2711a88 P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.319866 10257 tablet_replica.cc:333] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.320348 10257 raft_consensus.cc:2243] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.320734 10257 raft_consensus.cc:2272] T a5dc85cb531340ffa57355c8424590bc P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.322683 10257 tablet_replica.cc:333] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.323225 10257 raft_consensus.cc:2243] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.324009 10257 raft_consensus.cc:2272] T ab534676680e4b1184abab27ddb6752c P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.325866 10257 tablet_replica.cc:333] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.326350 10257 raft_consensus.cc:2243] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.326757 10257 raft_consensus.cc:2272] T 8db17f5622b84012a65f30d5ece1d98c P 4cd696c5af034cc098882be56fff7076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.328766 10257 tablet_replica.cc:333] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.329277 10257 raft_consensus.cc:2243] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.329679 10257 raft_consensus.cc:2272] T 964f1f17376d48538f5f1827dba39ee5 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.331707 10257 tablet_replica.cc:333] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.332288 10257 raft_consensus.cc:2243] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.333077 10257 raft_consensus.cc:2272] T 259855087e02457cb10e70f2808110b8 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.335346 10257 tablet_replica.cc:333] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.335951 10257 raft_consensus.cc:2243] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.336371 10257 raft_consensus.cc:2272] T 0dfc7b968a2a42dc92565aa8335ea320 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.338259 10257 tablet_replica.cc:333] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.338770 10257 raft_consensus.cc:2243] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.339613 10257 raft_consensus.cc:2272] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.341845 10257 tablet_replica.cc:333] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.342406 10257 raft_consensus.cc:2243] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.342883 10257 raft_consensus.cc:2272] T c0427a1799c24e9cb0546a29f73b1714 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.345098 10257 tablet_replica.cc:333] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.345660 10257 raft_consensus.cc:2243] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.346112 10257 raft_consensus.cc:2272] T a87f179730e34788b5ebbcebdd11741e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.348403 10257 tablet_replica.cc:333] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.349014 10257 raft_consensus.cc:2243] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.349468 10257 raft_consensus.cc:2272] T e14b49abcadf494fb27aca35637c2bd6 P 4cd696c5af034cc098882be56fff7076 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.351688 10257 tablet_replica.cc:333] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.352190 10257 raft_consensus.cc:2243] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.352677 10257 raft_consensus.cc:2272] T f281ecea50c24327b8006c8f08c72736 P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.354679 10257 tablet_replica.cc:333] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076: stopping tablet replica
I20260709 13:18:49.355124 10257 raft_consensus.cc:2243] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.355685 10257 raft_consensus.cc:2272] T b6ac007579544fe196e9a79834912e9e P 4cd696c5af034cc098882be56fff7076 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:49.364914 20419 consensus_peers.cc:597] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 -> Peer 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101): Couldn't send request to peer 4cd696c5af034cc098882be56fff7076. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.397709 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
W20260709 13:18:49.433284 20419 consensus_peers.cc:597] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 -> Peer 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101): Couldn't send request to peer 4cd696c5af034cc098882be56fff7076. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:39101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.436034 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
W20260709 13:18:49.442219 20419 consensus_peers.cc:597] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 -> Peer 4cd696c5af034cc098882be56fff7076 (127.10.4.66:39101): Couldn't send request to peer 4cd696c5af034cc098882be56fff7076. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:39101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:49.458091 20418 consensus_peers.cc:597] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 -> Peer ec3890a9489c444385a32dde2824973f (127.10.4.65:44339): Couldn't send request to peer ec3890a9489c444385a32dde2824973f. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:44339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.464377 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:49.465137 10257 tablet_replica.cc:333] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.465834 10257 raft_consensus.cc:2243] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.466661 10257 raft_consensus.cc:2272] T 4a144f8a86ec4b9982e6c06ac872b1fd P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.468833 10257 tablet_replica.cc:333] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.469484 10257 raft_consensus.cc:2243] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.469978 10257 raft_consensus.cc:2272] T a258a503dee046afbebc36a2d850f541 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.471968 10257 tablet_replica.cc:333] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.472481 10257 raft_consensus.cc:2243] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.473285 10257 raft_consensus.cc:2272] T c7bb9c20fc774e248068eb0fb922884d P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.475562 10257 tablet_replica.cc:333] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.476125 10257 raft_consensus.cc:2243] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.476857 10257 raft_consensus.cc:2272] T a5dc85cb531340ffa57355c8424590bc P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.478775 10257 tablet_replica.cc:333] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.479318 10257 raft_consensus.cc:2243] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.479777 10257 raft_consensus.cc:2272] T ab534676680e4b1184abab27ddb6752c P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.481873 10257 tablet_replica.cc:333] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.482421 10257 raft_consensus.cc:2243] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.482827 10257 raft_consensus.cc:2272] T 8db17f5622b84012a65f30d5ece1d98c P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.484673 10257 tablet_replica.cc:333] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.485213 10257 raft_consensus.cc:2243] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.485909 10257 raft_consensus.cc:2272] T 964f1f17376d48538f5f1827dba39ee5 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.487762 10257 tablet_replica.cc:333] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.488288 10257 raft_consensus.cc:2243] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.488698 10257 raft_consensus.cc:2272] T 259855087e02457cb10e70f2808110b8 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.490597 10257 tablet_replica.cc:333] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.491107 10257 raft_consensus.cc:2243] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.491541 10257 raft_consensus.cc:2272] T c0427a1799c24e9cb0546a29f73b1714 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.493485 10257 tablet_replica.cc:333] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.493974 10257 raft_consensus.cc:2243] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.494397 10257 raft_consensus.cc:2272] T 0dfc7b968a2a42dc92565aa8335ea320 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.496308 10257 tablet_replica.cc:333] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.496816 10257 raft_consensus.cc:2243] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.497285 10257 raft_consensus.cc:2272] T 0cd187256e1c4ee3a19a1f4ee62d3dea P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.499284 10257 tablet_replica.cc:333] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.499805 10257 raft_consensus.cc:2243] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.500602 10257 raft_consensus.cc:2272] T a87f179730e34788b5ebbcebdd11741e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.502629 10257 tablet_replica.cc:333] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.503163 10257 raft_consensus.cc:2243] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.503931 10257 raft_consensus.cc:2272] T e14b49abcadf494fb27aca35637c2bd6 P 4c7e3a8fef9c47b5b879adb450333905 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.505842 10257 tablet_replica.cc:333] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.506314 10257 raft_consensus.cc:2243] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.506708 10257 raft_consensus.cc:2272] T 2e773ebd3ba0400181d5d44339c22cd1 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.508601 10257 tablet_replica.cc:333] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.509285 10257 raft_consensus.cc:2243] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.509748 10257 raft_consensus.cc:2272] T c8717bb64ae04786857242f5653cc8ee P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.511600 10257 tablet_replica.cc:333] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.512048 10257 raft_consensus.cc:2243] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.512413 10257 raft_consensus.cc:2272] T b2b5471415d545cd8e6fc1c4f2711a88 P 4c7e3a8fef9c47b5b879adb450333905 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.514394 10257 tablet_replica.cc:333] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.514885 10257 raft_consensus.cc:2243] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.515316 10257 raft_consensus.cc:2272] T f281ecea50c24327b8006c8f08c72736 P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.517269 10257 tablet_replica.cc:333] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905: stopping tablet replica
I20260709 13:18:49.517753 10257 raft_consensus.cc:2243] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:49.518172 10257 raft_consensus.cc:2272] T b6ac007579544fe196e9a79834912e9e P 4c7e3a8fef9c47b5b879adb450333905 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.556874 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
I20260709 13:18:49.588996 10257 master.cc:562] Master@127.10.4.126:45777 shutting down...
I20260709 13:18:49.610301 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:49.610846 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:49.611171 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P 98e48a1d079940ab9965431b63a5bbaa: stopping tablet replica
I20260709 13:18:49.631292 10257 master.cc:584] Master@127.10.4.126:45777 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12278 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:18:49.682467 10257 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.4.126:45619
I20260709 13:18:49.683794 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:49.689606 20764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:49.689842 20765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:49.690127 20767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:49.690775 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:49.691663 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:49.691835 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:49.691988 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603129691968 us; error 0 us; skew 500 ppm
I20260709 13:18:49.692493 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:49.694828 10257 webserver.cc:533] Webserver started at http://127.10.4.126:39273/ using document root <none> and password file <none>
I20260709 13:18:49.695277 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:49.695468 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:49.695762 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:49.696815 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/master-0-root/instance:
uuid: "4442319c201049d98bc8b46cb5407e9b"
format_stamp: "Formatted at 2026-07-09 13:18:49 on dist-test-slave-gkm9"
I20260709 13:18:49.701036 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:18:49.704335 20772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:49.705017 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:49.705288 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/master-0-root
uuid: "4442319c201049d98bc8b46cb5407e9b"
format_stamp: "Formatted at 2026-07-09 13:18:49 on dist-test-slave-gkm9"
I20260709 13:18:49.705551 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:49.732401 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:49.733593 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:49.770061 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.126:45619
I20260709 13:18:49.770182 20823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.126:45619 every 8 connection(s)
I20260709 13:18:49.773895 20824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:49.785342 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b: Bootstrap starting.
I20260709 13:18:49.790460 20824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:49.795125 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b: No bootstrap required, opened a new log
I20260709 13:18:49.797415 20824 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4442319c201049d98bc8b46cb5407e9b" member_type: VOTER }
I20260709 13:18:49.797796 20824 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:49.798020 20824 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4442319c201049d98bc8b46cb5407e9b, State: Initialized, Role: FOLLOWER
I20260709 13:18:49.798628 20824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4442319c201049d98bc8b46cb5407e9b" member_type: VOTER }
I20260709 13:18:49.799084 20824 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:18:49.799294 20824 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:18:49.799551 20824 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:49.805071 20824 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4442319c201049d98bc8b46cb5407e9b" member_type: VOTER }
I20260709 13:18:49.805637 20824 leader_election.cc:304] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4442319c201049d98bc8b46cb5407e9b; no voters: 
I20260709 13:18:49.806857 20824 leader_election.cc:290] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:18:49.807222 20827 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:49.808614 20827 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 1 LEADER]: Becoming Leader. State: Replica: 4442319c201049d98bc8b46cb5407e9b, State: Running, Role: LEADER
I20260709 13:18:49.809291 20827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4442319c201049d98bc8b46cb5407e9b" member_type: VOTER }
I20260709 13:18:49.809893 20824 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:18:49.813459 20828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4442319c201049d98bc8b46cb5407e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4442319c201049d98bc8b46cb5407e9b" member_type: VOTER } }
I20260709 13:18:49.814208 20828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:49.814260 20829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4442319c201049d98bc8b46cb5407e9b. Latest consensus state: current_term: 1 leader_uuid: "4442319c201049d98bc8b46cb5407e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4442319c201049d98bc8b46cb5407e9b" member_type: VOTER } }
I20260709 13:18:49.815025 20829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:49.817401 20834 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:18:49.822247 20834 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:18:49.826591 10257 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:18:49.830672 20834 catalog_manager.cc:1383] Generated new cluster ID: 6430e182b3a04960b5d8b5b8757227a0
I20260709 13:18:49.830937 20834 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:18:49.847859 20834 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:18:49.849097 20834 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:18:49.862453 20834 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b: Generated new TSK 0
I20260709 13:18:49.863018 20834 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:18:49.894559 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:49.901163 20845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:49.902642 20846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:49.905761 20848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:49.906595 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:49.907610 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:49.907804 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:49.907963 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603129907946 us; error 0 us; skew 500 ppm
I20260709 13:18:49.908445 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:49.910701 10257 webserver.cc:533] Webserver started at http://127.10.4.65:36243/ using document root <none> and password file <none>
I20260709 13:18:49.911257 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:49.911461 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:49.911720 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:49.912776 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-0-root/instance:
uuid: "5b97fb429b274b6087bc1de914d8f839"
format_stamp: "Formatted at 2026-07-09 13:18:49 on dist-test-slave-gkm9"
I20260709 13:18:49.917109 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:18:49.920307 20853 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:49.921059 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:49.921334 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-0-root
uuid: "5b97fb429b274b6087bc1de914d8f839"
format_stamp: "Formatted at 2026-07-09 13:18:49 on dist-test-slave-gkm9"
I20260709 13:18:49.921648 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:49.930473 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:49.931732 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:49.933121 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:49.935272 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:49.935487 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:49.935702 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:49.935866 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:49.973413 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.65:41611
I20260709 13:18:49.973534 20915 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.65:41611 every 8 connection(s)
I20260709 13:18:49.978048 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:49.985085 20920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:49.986999 20921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:49.989545 20923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:49.989763 10257 server_base.cc:1061] running on GCE node
I20260709 13:18:49.990774 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:49.991019 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:49.991217 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603129991199 us; error 0 us; skew 500 ppm
I20260709 13:18:49.991753 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:49.994349 10257 webserver.cc:533] Webserver started at http://127.10.4.66:39921/ using document root <none> and password file <none>
I20260709 13:18:49.994798 20916 heartbeater.cc:344] Connected to a master server at 127.10.4.126:45619
I20260709 13:18:49.994901 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:49.995240 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:49.995244 20916 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:49.995556 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:49.996098 20916 heartbeater.cc:507] Master 127.10.4.126:45619 requested a full tablet report, sending...
I20260709 13:18:49.996800 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-1-root/instance:
uuid: "977832f370f745c7a0cb21e3dca7d578"
format_stamp: "Formatted at 2026-07-09 13:18:49 on dist-test-slave-gkm9"
I20260709 13:18:49.998340 20789 ts_manager.cc:194] Registered new tserver with Master: 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611)
I20260709 13:18:50.001001 20789 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49128
I20260709 13:18:50.002651 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:50.006076 20928 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:50.006881 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:18:50.007174 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-1-root
uuid: "977832f370f745c7a0cb21e3dca7d578"
format_stamp: "Formatted at 2026-07-09 13:18:49 on dist-test-slave-gkm9"
I20260709 13:18:50.007478 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:50.017081 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:50.018144 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:50.019539 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:50.021646 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:50.021842 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:50.022048 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:50.022192 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:50.060781 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.66:43519
I20260709 13:18:50.060882 20990 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.66:43519 every 8 connection(s)
I20260709 13:18:50.065399 10257 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:50.072731 20994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:50.075248 20995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:50.075830 10257 server_base.cc:1061] running on GCE node
W20260709 13:18:50.076685 20997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:50.077626 10257 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:50.077831 10257 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:50.078030 20991 heartbeater.cc:344] Connected to a master server at 127.10.4.126:45619
I20260709 13:18:50.078027 10257 hybrid_clock.cc:648] HybridClock initialized: now 1783603130078007 us; error 0 us; skew 500 ppm
I20260709 13:18:50.078523 20991 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:50.078882 10257 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:50.079249 20991 heartbeater.cc:507] Master 127.10.4.126:45619 requested a full tablet report, sending...
I20260709 13:18:50.081311 20789 ts_manager.cc:194] Registered new tserver with Master: 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:50.081591 10257 webserver.cc:533] Webserver started at http://127.10.4.67:46435/ using document root <none> and password file <none>
I20260709 13:18:50.082118 10257 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:50.082291 10257 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:50.082526 10257 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:50.082716 20789 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49134
I20260709 13:18:50.083794 10257 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-2-root/instance:
uuid: "3106c79919c14b9b995f45270d557a1a"
format_stamp: "Formatted at 2026-07-09 13:18:50 on dist-test-slave-gkm9"
I20260709 13:18:50.088001 10257 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:18:50.091118 21002 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:50.091885 10257 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:18:50.092157 10257 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-2-root
uuid: "3106c79919c14b9b995f45270d557a1a"
format_stamp: "Formatted at 2026-07-09 13:18:50 on dist-test-slave-gkm9"
I20260709 13:18:50.092415 10257 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskP_YfA4/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602834947472-10257-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:50.120481 10257 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:50.121742 10257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:50.123104 10257 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:50.125319 10257 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:50.125506 10257 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:50.125741 10257 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:50.125886 10257 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:50.164604 10257 rpc_server.cc:307] RPC server started. Bound to: 127.10.4.67:35839
I20260709 13:18:50.164723 21064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.4.67:35839 every 8 connection(s)
I20260709 13:18:50.177380 21065 heartbeater.cc:344] Connected to a master server at 127.10.4.126:45619
I20260709 13:18:50.177809 21065 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:50.178490 21065 heartbeater.cc:507] Master 127.10.4.126:45619 requested a full tablet report, sending...
I20260709 13:18:50.180310 20789 ts_manager.cc:194] Registered new tserver with Master: 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.180877 10257 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013188771s
I20260709 13:18:50.181656 20789 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49148
I20260709 13:18:50.204612 20789 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49150:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:50.207113 20789 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:50.279551 20956 tablet_service.cc:1511] Processing CreateTablet for tablet d47c528029b94270a3e760ddcb2e6503 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:50.281113 20956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d47c528029b94270a3e760ddcb2e6503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.285450 20955 tablet_service.cc:1511] Processing CreateTablet for tablet aab942b20de44ae7bab51d2bf83142ac (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:50.286994 20955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab942b20de44ae7bab51d2bf83142ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.289217 20954 tablet_service.cc:1511] Processing CreateTablet for tablet dfa44f2202b74d6fa6853bff5d64d1d2 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:50.290541 20954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa44f2202b74d6fa6853bff5d64d1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.291817 21029 tablet_service.cc:1511] Processing CreateTablet for tablet aab942b20de44ae7bab51d2bf83142ac (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:50.292508 21027 tablet_service.cc:1511] Processing CreateTablet for tablet a8e2647fab8b4079861f111406365333 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:50.293334 21029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab942b20de44ae7bab51d2bf83142ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.293962 21028 tablet_service.cc:1511] Processing CreateTablet for tablet dfa44f2202b74d6fa6853bff5d64d1d2 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:50.295305 21028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa44f2202b74d6fa6853bff5d64d1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.296288 20953 tablet_service.cc:1511] Processing CreateTablet for tablet a8e2647fab8b4079861f111406365333 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:50.297607 21027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8e2647fab8b4079861f111406365333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.290473 21030 tablet_service.cc:1511] Processing CreateTablet for tablet d47c528029b94270a3e760ddcb2e6503 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:50.298776 21030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d47c528029b94270a3e760ddcb2e6503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.299324 20881 tablet_service.cc:1511] Processing CreateTablet for tablet d47c528029b94270a3e760ddcb2e6503 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:50.300702 20881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d47c528029b94270a3e760ddcb2e6503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.297662 20953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8e2647fab8b4079861f111406365333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.311825 20880 tablet_service.cc:1511] Processing CreateTablet for tablet aab942b20de44ae7bab51d2bf83142ac (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:50.313725 20880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab942b20de44ae7bab51d2bf83142ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.321988 20952 tablet_service.cc:1511] Processing CreateTablet for tablet b46ada670408442da2f3289f10ca33b3 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:50.323611 20952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46ada670408442da2f3289f10ca33b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.324712 21026 tablet_service.cc:1511] Processing CreateTablet for tablet b46ada670408442da2f3289f10ca33b3 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:50.326148 21026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46ada670408442da2f3289f10ca33b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.334300 20877 tablet_service.cc:1511] Processing CreateTablet for tablet b46ada670408442da2f3289f10ca33b3 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:50.335716 20877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46ada670408442da2f3289f10ca33b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.344751 21025 tablet_service.cc:1511] Processing CreateTablet for tablet 4761164a28c243c1ae1500d536445d9b (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:50.346232 21025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4761164a28c243c1ae1500d536445d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.348835 20879 tablet_service.cc:1511] Processing CreateTablet for tablet dfa44f2202b74d6fa6853bff5d64d1d2 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:50.350194 20879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa44f2202b74d6fa6853bff5d64d1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.353669 20878 tablet_service.cc:1511] Processing CreateTablet for tablet a8e2647fab8b4079861f111406365333 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:50.354993 20878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8e2647fab8b4079861f111406365333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.357730 20951 tablet_service.cc:1511] Processing CreateTablet for tablet 4761164a28c243c1ae1500d536445d9b (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:50.359107 20951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4761164a28c243c1ae1500d536445d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.359678 20875 tablet_service.cc:1511] Processing CreateTablet for tablet 2f1e56bf4cc5416ea22348bc387b57a9 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:50.361073 20875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1e56bf4cc5416ea22348bc387b57a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.364602 20876 tablet_service.cc:1511] Processing CreateTablet for tablet 4761164a28c243c1ae1500d536445d9b (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:50.366012 20876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4761164a28c243c1ae1500d536445d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.369601 20950 tablet_service.cc:1511] Processing CreateTablet for tablet 2f1e56bf4cc5416ea22348bc387b57a9 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:50.370959 20950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1e56bf4cc5416ea22348bc387b57a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.371971 21025 tablet_service.cc:1511] Processing CreateTablet for tablet 2f1e56bf4cc5416ea22348bc387b57a9 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:50.373313 21025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1e56bf4cc5416ea22348bc387b57a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.376725 20874 tablet_service.cc:1511] Processing CreateTablet for tablet a5d7f0637d564294bcd6920033ba9fbf (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:50.378101 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d7f0637d564294bcd6920033ba9fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.388504 20952 tablet_service.cc:1511] Processing CreateTablet for tablet a5d7f0637d564294bcd6920033ba9fbf (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:50.390056 20952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d7f0637d564294bcd6920033ba9fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.390163 21028 tablet_service.cc:1511] Processing CreateTablet for tablet a5d7f0637d564294bcd6920033ba9fbf (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:50.391695 21028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d7f0637d564294bcd6920033ba9fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.406441 20873 tablet_service.cc:1511] Processing CreateTablet for tablet e83e6552f0674e9698832dfbd218b118 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:50.407981 20873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83e6552f0674e9698832dfbd218b118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.409502 20949 tablet_service.cc:1511] Processing CreateTablet for tablet e83e6552f0674e9698832dfbd218b118 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:50.410605 21029 tablet_service.cc:1511] Processing CreateTablet for tablet e83e6552f0674e9698832dfbd218b118 (DEFAULT_TABLE table=test-workload [id=aa97bc15dbb9490fa0b5635b36c46c37]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:50.414007 21029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83e6552f0674e9698832dfbd218b118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.433988 20949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83e6552f0674e9698832dfbd218b118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.473505 21080 tablet_bootstrap.cc:492] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.473850 21081 tablet_bootstrap.cc:492] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.476627 21082 tablet_bootstrap.cc:492] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:50.490651 21082 tablet_bootstrap.cc:654] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.490655 21081 tablet_bootstrap.cc:654] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.496057 21080 tablet_bootstrap.cc:654] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.501639 21080 tablet_bootstrap.cc:492] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.502236 21080 ts_tablet_manager.cc:1403] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.006s
I20260709 13:18:50.505209 21080 raft_consensus.cc:359] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.506124 21080 raft_consensus.cc:385] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.506484 21080 raft_consensus.cc:740] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.507519 21080 consensus_queue.cc:260] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.510181 21081 tablet_bootstrap.cc:492] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.510349 20991 heartbeater.cc:499] Master 127.10.4.126:45619 was elected leader, sending a full tablet report...
I20260709 13:18:50.510787 21082 tablet_bootstrap.cc:492] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:50.511018 21081 ts_tablet_manager.cc:1403] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.028s
I20260709 13:18:50.511152 21080 ts_tablet_manager.cc:1434] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:18:50.511475 21082 ts_tablet_manager.cc:1403] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.035s	user 0.025s	sys 0.005s
I20260709 13:18:50.512434 21080 tablet_bootstrap.cc:492] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.514439 21081 raft_consensus.cc:359] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.515302 21081 raft_consensus.cc:385] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.514604 21082 raft_consensus.cc:359] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:50.515683 21081 raft_consensus.cc:740] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.516115 21082 raft_consensus.cc:385] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.516475 21082 raft_consensus.cc:740] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.516700 21081 consensus_queue.cc:260] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.517433 21082 consensus_queue.cc:260] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:50.519083 21080 tablet_bootstrap.cc:654] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.523237 20916 heartbeater.cc:499] Master 127.10.4.126:45619 was elected leader, sending a full tablet report...
I20260709 13:18:50.524595 21082 ts_tablet_manager.cc:1434] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:18:50.526018 21082 tablet_bootstrap.cc:492] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:50.533768 21082 tablet_bootstrap.cc:654] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.535218 21065 heartbeater.cc:499] Master 127.10.4.126:45619 was elected leader, sending a full tablet report...
I20260709 13:18:50.535753 21081 ts_tablet_manager.cc:1434] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.024s	user 0.014s	sys 0.010s
I20260709 13:18:50.536211 21080 tablet_bootstrap.cc:492] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.536748 21080 ts_tablet_manager.cc:1403] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260709 13:18:50.536932 21081 tablet_bootstrap.cc:492] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.539626 21080 raft_consensus.cc:359] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.540454 21080 raft_consensus.cc:385] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.540803 21080 raft_consensus.cc:740] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.541692 21080 consensus_queue.cc:260] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.543707 21081 tablet_bootstrap.cc:654] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.543931 21080 ts_tablet_manager.cc:1434] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:50.546090 21080 tablet_bootstrap.cc:492] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.553608 21080 tablet_bootstrap.cc:654] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.562682 21081 tablet_bootstrap.cc:492] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.563465 21081 ts_tablet_manager.cc:1403] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.000s
I20260709 13:18:50.563642 21082 tablet_bootstrap.cc:492] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:50.564396 21082 ts_tablet_manager.cc:1403] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.039s	user 0.002s	sys 0.014s
I20260709 13:18:50.569259 21082 raft_consensus.cc:359] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.571734 21082 raft_consensus.cc:385] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.571354 21081 raft_consensus.cc:359] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.572129 21082 raft_consensus.cc:740] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.572221 21081 raft_consensus.cc:385] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.572659 21081 raft_consensus.cc:740] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.573066 21080 tablet_bootstrap.cc:492] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.573148 21082 consensus_queue.cc:260] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.573724 21080 ts_tablet_manager.cc:1403] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.013s
I20260709 13:18:50.573575 21081 consensus_queue.cc:260] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.575832 21081 ts_tablet_manager.cc:1434] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:18:50.576812 21081 tablet_bootstrap.cc:492] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.576968 21080 raft_consensus.cc:359] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.577986 21080 raft_consensus.cc:385] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.578336 21080 raft_consensus.cc:740] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.579200 21080 consensus_queue.cc:260] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.579639 21082 ts_tablet_manager.cc:1434] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 13:18:50.580787 21082 tablet_bootstrap.cc:492] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:50.581372 21080 ts_tablet_manager.cc:1434] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:50.583875 21086 raft_consensus.cc:493] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.584336 21086 raft_consensus.cc:515] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.584364 21081 tablet_bootstrap.cc:654] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.591082 21086 leader_election.cc:290] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.593276 21082 tablet_bootstrap.cc:654] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.593711 21086 raft_consensus.cc:493] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.593832 21087 raft_consensus.cc:493] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.595458 21088 raft_consensus.cc:493] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.595616 21087 raft_consensus.cc:515] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.596035 21088 raft_consensus.cc:515] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.598855 21088 leader_election.cc:290] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.604707 21087 leader_election.cc:290] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:50.606030 21081 tablet_bootstrap.cc:492] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.606714 21081 ts_tablet_manager.cc:1403] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.004s
I20260709 13:18:50.612898 21086 raft_consensus.cc:515] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.615368 21086 leader_election.cc:290] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.615929 21081 raft_consensus.cc:359] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.622146 21081 raft_consensus.cc:385] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.623441 21080 tablet_bootstrap.cc:492] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.624387 21081 raft_consensus.cc:740] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.636153 21081 consensus_queue.cc:260] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.637559 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8e2647fab8b4079861f111406365333" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:50.638406 21081 ts_tablet_manager.cc:1434] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.002s
I20260709 13:18:50.639575 21081 tablet_bootstrap.cc:492] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.639652 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:50.640460 21040 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:50.641132 21080 tablet_bootstrap.cc:654] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.641894 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
W20260709 13:18:50.642884 20932 leader_election.cc:343] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.644057 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:50.645421 20854 leader_election.cc:304] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:50.646329 21081 tablet_bootstrap.cc:654] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.646402 21088 raft_consensus.cc:2804] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:50.646919 21088 raft_consensus.cc:493] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.647235 21088 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.653803 21080 tablet_bootstrap.cc:492] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.654497 21088 raft_consensus.cc:515] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
W20260709 13:18:50.655604 20932 leader_election.cc:343] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:50.656911 21006 leader_election.cc:343] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.657945 21080 ts_tablet_manager.cc:1403] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.003s
I20260709 13:18:50.658900 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:50.659552 21040 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.666736 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8e2647fab8b4079861f111406365333" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:50.666966 21040 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:50.657205 21088 leader_election.cc:290] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.667498 21039 raft_consensus.cc:2468] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:50.668529 20854 leader_election.cc:304] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:50.669413 21088 raft_consensus.cc:2804] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:50.669314 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:50.671598 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:50.672307 20966 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:50.662473 21080 raft_consensus.cc:359] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.673870 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:50.674386 21080 raft_consensus.cc:385] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.674573 21081 tablet_bootstrap.cc:492] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.674551 20966 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.675165 21080 raft_consensus.cc:740] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.675946 21080 consensus_queue.cc:260] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.675361 20929 leader_election.cc:304] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:50.678145 21080 ts_tablet_manager.cc:1434] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.000s
I20260709 13:18:50.679149 21080 tablet_bootstrap.cc:492] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.682345 20966 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:50.683143 21081 ts_tablet_manager.cc:1403] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.019s
I20260709 13:18:50.685353 21080 tablet_bootstrap.cc:654] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.686672 21081 raft_consensus.cc:359] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.687495 21081 raft_consensus.cc:385] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.687848 21081 raft_consensus.cc:740] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.688794 21081 consensus_queue.cc:260] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.691180 21081 ts_tablet_manager.cc:1434] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:18:50.692369 21081 tablet_bootstrap.cc:492] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.693954 21087 raft_consensus.cc:493] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.694537 21087 raft_consensus.cc:515] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.698271 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:50.699215 21086 raft_consensus.cc:2804] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:18:50.699736 20929 leader_election.cc:343] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.700327 20929 leader_election.cc:304] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 977832f370f745c7a0cb21e3dca7d578; no voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:50.701186 21096 raft_consensus.cc:2749] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:50.697419 21087 leader_election.cc:290] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
W20260709 13:18:50.703589 21006 leader_election.cc:343] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.709350 21081 tablet_bootstrap.cc:654] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.709692 21080 tablet_bootstrap.cc:492] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.710266 21080 ts_tablet_manager.cc:1403] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.016s
I20260709 13:18:50.699756 21086 raft_consensus.cc:493] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.712321 21086 raft_consensus.cc:3060] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.713500 21082 tablet_bootstrap.cc:492] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:50.714074 21082 ts_tablet_manager.cc:1403] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.134s	user 0.003s	sys 0.019s
I20260709 13:18:50.717432 21082 raft_consensus.cc:359] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.718374 21082 raft_consensus.cc:385] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.718854 21082 raft_consensus.cc:740] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.719899 21082 consensus_queue.cc:260] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.725755 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:50.725824 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:50.727459 21086 raft_consensus.cc:515] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.728271 21082 ts_tablet_manager.cc:1434] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.002s
I20260709 13:18:50.729341 21082 tablet_bootstrap.cc:492] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:50.727232 21088 raft_consensus.cc:697] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:50.731258 20965 raft_consensus.cc:2468] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 0.
I20260709 13:18:50.731465 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8e2647fab8b4079861f111406365333" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:50.730290 21086 leader_election.cc:290] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.730756 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8e2647fab8b4079861f111406365333" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:50.732286 21080 raft_consensus.cc:359] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:50.732797 21039 raft_consensus.cc:3060] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.733032 21080 raft_consensus.cc:385] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:18:50.733228 20932 leader_election.cc:343] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.734606 21081 tablet_bootstrap.cc:492] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.735042 21081 ts_tablet_manager.cc:1403] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.008s
I20260709 13:18:50.735502 21080 raft_consensus.cc:740] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.736418 21080 consensus_queue.cc:260] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:50.737998 21081 raft_consensus.cc:359] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.738651 21082 tablet_bootstrap.cc:654] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.738752 21081 raft_consensus.cc:385] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:18:50.730381 21004 leader_election.cc:343] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.740688 21088 consensus_queue.cc:237] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.742219 21080 ts_tablet_manager.cc:1434] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 13:18:50.740049 21039 raft_consensus.cc:2468] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:50.743251 21080 tablet_bootstrap.cc:492] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.744210 20929 leader_election.cc:304] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:50.745850 21086 raft_consensus.cc:2804] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:50.739487 21106 raft_consensus.cc:493] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.746412 21086 raft_consensus.cc:697] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:50.746815 21106 raft_consensus.cc:515] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.747771 21086 consensus_queue.cc:237] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.749643 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:50.750195 21082 tablet_bootstrap.cc:492] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:50.750685 21082 ts_tablet_manager.cc:1403] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.012s
W20260709 13:18:50.751073 20855 leader_election.cc:343] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.740006 21004 leader_election.cc:304] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a; no voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:50.752395 21004 leader_election.cc:304] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:50.753280 21087 raft_consensus.cc:2749] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:50.753754 21080 tablet_bootstrap.cc:654] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.753914 21087 raft_consensus.cc:2804] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:50.754282 21087 raft_consensus.cc:493] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.754591 21087 raft_consensus.cc:3060] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.755790 21082 raft_consensus.cc:359] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.756547 21082 raft_consensus.cc:385] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.756819 21082 raft_consensus.cc:740] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.757579 21082 consensus_queue.cc:260] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.761773 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:50.762814 21082 ts_tablet_manager.cc:1434] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
W20260709 13:18:50.763546 20854 leader_election.cc:343] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.764022 20854 leader_election.cc:304] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839; no voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:50.764823 21088 raft_consensus.cc:2749] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:50.764865 21106 leader_election.cc:290] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.765192 21080 tablet_bootstrap.cc:492] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.765905 21080 ts_tablet_manager.cc:1403] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.004s
I20260709 13:18:50.767386 20786 catalog_manager.cc:5697] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:50.768985 21080 raft_consensus.cc:359] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.769740 21080 raft_consensus.cc:385] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.770241 21080 raft_consensus.cc:740] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.771287 21080 consensus_queue.cc:260] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.772449 21082 tablet_bootstrap.cc:492] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:50.773990 21080 ts_tablet_manager.cc:1434] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:50.775072 21080 tablet_bootstrap.cc:492] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.776541 21081 raft_consensus.cc:740] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.777343 21081 consensus_queue.cc:260] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.779445 21082 tablet_bootstrap.cc:654] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.780049 21087 raft_consensus.cc:515] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.781378 21080 tablet_bootstrap.cc:654] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.782961 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
W20260709 13:18:50.784538 21006 leader_election.cc:343] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.783028 20787 catalog_manager.cc:5697] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:50.787901 21087 leader_election.cc:290] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:50.791427 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:50.792069 20965 raft_consensus.cc:3060] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.796217 21080 tablet_bootstrap.cc:492] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.797122 21080 ts_tablet_manager.cc:1403] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.004s
I20260709 13:18:50.799983 21080 raft_consensus.cc:359] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.800707 21080 raft_consensus.cc:385] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.801028 21080 raft_consensus.cc:740] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.801759 21080 consensus_queue.cc:260] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.803891 21080 ts_tablet_manager.cc:1434] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:50.803825 20965 raft_consensus.cc:2468] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 1.
I20260709 13:18:50.805110 21080 tablet_bootstrap.cc:492] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.796604 21081 ts_tablet_manager.cc:1434] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.001s
I20260709 13:18:50.808009 21081 tablet_bootstrap.cc:492] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.805248 21004 leader_election.cc:304] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:50.811997 21080 tablet_bootstrap.cc:654] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.815136 21087 raft_consensus.cc:2804] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:50.815811 21087 raft_consensus.cc:697] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:50.817173 21087 consensus_queue.cc:237] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
W20260709 13:18:50.823506 20837 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:50.823741 21082 tablet_bootstrap.cc:492] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:50.824420 21082 ts_tablet_manager.cc:1403] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.026s
I20260709 13:18:50.827893 21082 raft_consensus.cc:359] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.828621 21082 raft_consensus.cc:385] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.828971 21082 raft_consensus.cc:740] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.829592 21081 tablet_bootstrap.cc:654] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.830220 21080 tablet_bootstrap.cc:492] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.829995 21082 consensus_queue.cc:260] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.830773 21080 ts_tablet_manager.cc:1403] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.007s
I20260709 13:18:50.832954 21082 ts_tablet_manager.cc:1434] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:50.833665 21080 raft_consensus.cc:359] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.834138 21082 tablet_bootstrap.cc:492] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:50.834251 21086 raft_consensus.cc:493] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.834441 21080 raft_consensus.cc:385] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.834908 21080 raft_consensus.cc:740] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.834785 21086 raft_consensus.cc:515] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.835949 21080 consensus_queue.cc:260] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.836971 21086 leader_election.cc:290] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.838047 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:50.838555 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:50.839280 21039 raft_consensus.cc:2468] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
W20260709 13:18:50.839728 20932 leader_election.cc:343] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.840677 20929 leader_election.cc:304] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:50.841848 21086 raft_consensus.cc:2804] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:50.842077 21080 ts_tablet_manager.cc:1434] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:18:50.842309 21086 raft_consensus.cc:493] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.843139 21082 tablet_bootstrap.cc:654] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.843264 21080 tablet_bootstrap.cc:492] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:50.844980 21081 tablet_bootstrap.cc:492] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.845610 21081 ts_tablet_manager.cc:1403] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:18:50.842779 21086 raft_consensus.cc:3060] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.850044 21080 tablet_bootstrap.cc:654] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.853890 21081 raft_consensus.cc:359] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.854719 21081 raft_consensus.cc:385] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.855011 21081 raft_consensus.cc:740] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.855927 21081 consensus_queue.cc:260] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.858413 21081 ts_tablet_manager.cc:1434] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:18:50.861930 21086 raft_consensus.cc:515] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.864758 21081 tablet_bootstrap.cc:492] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.866369 21080 tablet_bootstrap.cc:492] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:50.866954 21080 ts_tablet_manager.cc:1403] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 13:18:50.866791 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:50.867630 20788 catalog_manager.cc:5697] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 1 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:50.868441 20932 leader_election.cc:343] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.870960 21081 tablet_bootstrap.cc:654] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.871650 21080 raft_consensus.cc:359] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.872398 21080 raft_consensus.cc:385] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.872721 21080 raft_consensus.cc:740] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.873447 21080 consensus_queue.cc:260] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.875540 21080 ts_tablet_manager.cc:1434] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:18:50.881999 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:50.882871 21039 raft_consensus.cc:3060] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.888283 21081 tablet_bootstrap.cc:492] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.888167 21086 leader_election.cc:290] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.890044 21081 ts_tablet_manager.cc:1403] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.006s
I20260709 13:18:50.890955 21082 tablet_bootstrap.cc:492] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:50.891621 21082 ts_tablet_manager.cc:1403] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.058s	user 0.028s	sys 0.018s
I20260709 13:18:50.892783 21081 raft_consensus.cc:359] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.893499 21081 raft_consensus.cc:385] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.895537 21081 raft_consensus.cc:740] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.894580 21082 raft_consensus.cc:359] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.895022 21039 raft_consensus.cc:2468] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:50.896232 21082 raft_consensus.cc:385] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.896567 21082 raft_consensus.cc:740] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.897169 20929 leader_election.cc:304] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:50.897089 21081 consensus_queue.cc:260] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.897385 21082 consensus_queue.cc:260] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.898056 21086 raft_consensus.cc:2804] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:50.898564 21086 raft_consensus.cc:697] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:50.899442 21086 consensus_queue.cc:237] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.900019 21081 ts_tablet_manager.cc:1434] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:18:50.901187 21081 tablet_bootstrap.cc:492] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.905306 21082 ts_tablet_manager.cc:1434] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 13:18:50.906818 21082 tablet_bootstrap.cc:492] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:50.907902 21081 tablet_bootstrap.cc:654] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.907804 21096 raft_consensus.cc:493] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.908706 21096 raft_consensus.cc:515] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.912497 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7f0637d564294bcd6920033ba9fbf" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:50.913309 20891 raft_consensus.cc:2468] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:50.913736 21082 tablet_bootstrap.cc:654] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.914834 20932 leader_election.cc:304] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:50.917032 21096 leader_election.cc:290] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.917150 21129 raft_consensus.cc:2804] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:50.919193 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7f0637d564294bcd6920033ba9fbf" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:50.920017 21039 raft_consensus.cc:2468] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:50.921916 21129 raft_consensus.cc:493] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.923542 21129 raft_consensus.cc:3060] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.929591 21081 tablet_bootstrap.cc:492] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:50.929402 21088 raft_consensus.cc:493] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.930380 21088 raft_consensus.cc:515] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:50.932778 21088 leader_election.cc:290] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.933656 21081 ts_tablet_manager.cc:1403] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.005s
I20260709 13:18:50.934346 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d47c528029b94270a3e760ddcb2e6503" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:50.935141 21087 raft_consensus.cc:493] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.935761 21087 raft_consensus.cc:515] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.937341 20786 catalog_manager.cc:5697] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:50.938767 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:50.938835 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d47c528029b94270a3e760ddcb2e6503" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:50.935027 20965 raft_consensus.cc:2468] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:50.940227 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:50.940817 21087 leader_election.cc:290] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:50.940968 20966 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 0.
I20260709 13:18:50.939337 20891 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 0.
I20260709 13:18:50.942742 21004 leader_election.cc:304] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:50.940512 20855 leader_election.cc:304] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:50.943229 21081 raft_consensus.cc:359] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
W20260709 13:18:50.944090 20854 leader_election.cc:343] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.948153 21088 raft_consensus.cc:2804] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:50.948825 21129 raft_consensus.cc:515] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.949219 21087 raft_consensus.cc:2804] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:50.948707 21088 raft_consensus.cc:493] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.950095 21088 raft_consensus.cc:3060] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.951237 21129 leader_election.cc:290] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.952127 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7f0637d564294bcd6920033ba9fbf" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:50.952817 20891 raft_consensus.cc:3060] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.952966 21087 raft_consensus.cc:493] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.953091 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7f0637d564294bcd6920033ba9fbf" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:50.953433 21087 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.953781 21039 raft_consensus.cc:3060] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.944099 21081 raft_consensus.cc:385] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.955281 21081 raft_consensus.cc:740] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.956032 21081 consensus_queue.cc:260] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.958038 21081 ts_tablet_manager.cc:1434] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.000s
I20260709 13:18:50.958151 21088 raft_consensus.cc:515] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:50.962122 20891 raft_consensus.cc:2468] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:50.963352 21039 raft_consensus.cc:2468] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:50.963356 20932 leader_election.cc:304] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:50.964593 21129 raft_consensus.cc:2804] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:50.965248 21129 raft_consensus.cc:697] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:50.965435 21081 tablet_bootstrap.cc:492] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:50.965368 21087 raft_consensus.cc:515] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.967592 21087 leader_election.cc:290] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:50.970039 21113 raft_consensus.cc:493] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.970316 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:50.971022 20966 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.970912 21113 raft_consensus.cc:515] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.972168 21081 tablet_bootstrap.cc:654] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:50.973284 21113 leader_election.cc:290] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:50.969650 21129 consensus_queue.cc:237] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.969502 21082 tablet_bootstrap.cc:492] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:50.975351 21082 ts_tablet_manager.cc:1403] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.015s
I20260709 13:18:50.976747 21088 leader_election.cc:290] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:50.975169 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:50.977509 20965 raft_consensus.cc:2468] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 0.
I20260709 13:18:50.979071 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d47c528029b94270a3e760ddcb2e6503" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:50.979290 21004 leader_election.cc:304] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:50.979734 20965 raft_consensus.cc:3060] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.981784 21113 raft_consensus.cc:2804] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:50.982144 21113 raft_consensus.cc:493] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.982442 21113 raft_consensus.cc:3060] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:50.984171 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d47c528029b94270a3e760ddcb2e6503" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
W20260709 13:18:50.987774 20854 leader_election.cc:343] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:50.988047 20890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:50.988902 20890 raft_consensus.cc:2468] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 0.
I20260709 13:18:50.989705 20965 raft_consensus.cc:2468] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:50.991555 21088 raft_consensus.cc:493] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.991665 20855 leader_election.cc:304] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 3106c79919c14b9b995f45270d557a1a
I20260709 13:18:50.992728 21106 raft_consensus.cc:2804] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:50.993186 21106 raft_consensus.cc:697] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:50.994035 21106 consensus_queue.cc:237] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:50.995460 20966 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 1.
I20260709 13:18:50.978932 21082 raft_consensus.cc:359] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.996654 21004 leader_election.cc:304] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:50.997040 21082 raft_consensus.cc:385] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:50.997380 21082 raft_consensus.cc:740] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:50.998181 21087 raft_consensus.cc:2804] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:51.001855 21113 raft_consensus.cc:515] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.992251 21088 raft_consensus.cc:515] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:50.998307 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:51.003438 21082 consensus_queue.cc:260] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.003970 20891 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.005918 21082 ts_tablet_manager.cc:1434] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:18:51.006042 21087 raft_consensus.cc:697] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:51.007119 21087 consensus_queue.cc:237] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.010783 20890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:51.006935 21082 tablet_bootstrap.cc:492] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:51.011351 20890 raft_consensus.cc:3060] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.015125 20891 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 1.
I20260709 13:18:51.017715 21082 tablet_bootstrap.cc:654] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:51.018843 20890 raft_consensus.cc:2468] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 1.
I20260709 13:18:51.020398 21081 tablet_bootstrap.cc:492] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:51.020709 21006 leader_election.cc:304] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:51.021081 21081 ts_tablet_manager.cc:1403] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.037s
I20260709 13:18:51.021510 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:51.021900 21134 raft_consensus.cc:2804] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:51.021716 20788 catalog_manager.cc:5697] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:51.021122 20786 catalog_manager.cc:5697] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.026533 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:51.024991 21113 leader_election.cc:290] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:51.026378 21088 leader_election.cc:290] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.025285 21081 raft_consensus.cc:359] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:51.022181 20966 raft_consensus.cc:2468] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:51.028276 21081 raft_consensus.cc:385] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:51.028661 21081 raft_consensus.cc:740] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:51.029421 20855 leader_election.cc:304] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:51.029569 21081 consensus_queue.cc:260] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:51.031572 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:51.032292 21081 ts_tablet_manager.cc:1434] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:18:51.032346 21088 raft_consensus.cc:2764] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:18:51.039222 21134 raft_consensus.cc:697] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:51.040457 21134 consensus_queue.cc:237] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.047277 20786 catalog_manager.cc:5697] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 1 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.048414 21082 tablet_bootstrap.cc:492] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:51.049264 21082 ts_tablet_manager.cc:1403] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.000s
I20260709 13:18:51.053752 21082 raft_consensus.cc:359] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.054291 21082 raft_consensus.cc:385] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:51.054548 21082 raft_consensus.cc:740] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:51.055083 21082 consensus_queue.cc:260] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.056766 21082 ts_tablet_manager.cc:1434] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:18:51.057857 21082 tablet_bootstrap.cc:492] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:51.062798 21082 tablet_bootstrap.cc:654] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:51.066541 20786 catalog_manager.cc:5697] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 1 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.067615 21082 tablet_bootstrap.cc:492] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:51.068220 21082 ts_tablet_manager.cc:1403] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:18:51.071108 21082 raft_consensus.cc:359] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.071907 21082 raft_consensus.cc:385] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:51.072212 21082 raft_consensus.cc:740] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:51.072911 21082 consensus_queue.cc:260] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.074908 21082 ts_tablet_manager.cc:1434] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:18:51.077217 21088 raft_consensus.cc:493] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.077900 21088 raft_consensus.cc:515] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.080318 21088 leader_election.cc:290] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.081198 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:51.081599 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4761164a28c243c1ae1500d536445d9b" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:51.082509 21039 raft_consensus.cc:2393] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b97fb429b274b6087bc1de914d8f839 in current term 1: Already voted for candidate 977832f370f745c7a0cb21e3dca7d578 in this term.
I20260709 13:18:51.083560 20854 leader_election.cc:304] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839; no voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:51.084266 21088 raft_consensus.cc:3060] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.089964 21088 raft_consensus.cc:2749] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:51.158233 21088 consensus_queue.cc:1048] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.170125 21088 consensus_queue.cc:1048] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.188100 20890 raft_consensus.cc:3060] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.194706 21129 consensus_queue.cc:1048] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.204967 21086 consensus_queue.cc:1048] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.277474 21086 raft_consensus.cc:493] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.277956 21086 raft_consensus.cc:515] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.280303 21086 leader_election.cc:290] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.280572 20890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:51.281277 20890 raft_consensus.cc:2468] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:51.281358 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:51.281961 21039 raft_consensus.cc:2468] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:51.282342 20891 raft_consensus.cc:3060] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.282226 20932 leader_election.cc:304] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:51.283099 21086 raft_consensus.cc:2804] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:51.283349 21086 raft_consensus.cc:493] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.283660 21086 raft_consensus.cc:3060] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.288738 21134 consensus_queue.cc:1048] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:51.288764 21086 raft_consensus.cc:515] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.290700 21086 leader_election.cc:290] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.291700 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:51.291891 20890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:51.292356 21039 raft_consensus.cc:3060] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.292397 20890 raft_consensus.cc:3060] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.299558 21039 raft_consensus.cc:2468] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:51.299558 20890 raft_consensus.cc:2468] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:51.301380 20929 leader_election.cc:304] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:51.302323 21086 raft_consensus.cc:2804] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:51.303568 21086 raft_consensus.cc:697] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:51.304425 21086 consensus_queue.cc:237] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.309669 21135 consensus_queue.cc:1048] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:51.316326 20787 catalog_manager.cc:5697] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:51.317454 21086 consensus_queue.cc:1048] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:51.330982 21129 consensus_queue.cc:1048] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.353690 21129 consensus_queue.cc:1048] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.363328 21096 consensus_queue.cc:1048] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.428925 21135 consensus_queue.cc:1048] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.439066 21087 consensus_queue.cc:1048] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.485503 21087 raft_consensus.cc:493] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.485944 21087 raft_consensus.cc:515] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } } peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } }
I20260709 13:18:51.487635 21087 leader_election.cc:290] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611)
I20260709 13:18:51.488492 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d47c528029b94270a3e760ddcb2e6503" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:51.488705 20890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d47c528029b94270a3e760ddcb2e6503" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:51.489193 20966 raft_consensus.cc:2393] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3106c79919c14b9b995f45270d557a1a in current term 1: Already voted for candidate 5b97fb429b274b6087bc1de914d8f839 in this term.
I20260709 13:18:51.490195 21006 leader_election.cc:304] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a; no voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:51.491056 21087 raft_consensus.cc:3060] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.496299 21087 raft_consensus.cc:2749] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:51.501166 21109 consensus_queue.cc:1048] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.508391 21088 consensus_queue.cc:1048] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.531666 21039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
mode: GRACEFUL
new_leader_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:57544
I20260709 13:18:51.532212 21039 raft_consensus.cc:606] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Received request to transfer leadership to TS 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:51.537693 21039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
mode: GRACEFUL
new_leader_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:57544
I20260709 13:18:51.538133 21039 raft_consensus.cc:606] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Received request to transfer leadership to TS 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:51.550390 20966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8e2647fab8b4079861f111406365333"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
mode: GRACEFUL
new_leader_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:60316
I20260709 13:18:51.550904 20966 raft_consensus.cc:606] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Received request to transfer leadership to TS 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:51.563172 20890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
mode: GRACEFUL
new_leader_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:60880
I20260709 13:18:51.563699 20890 raft_consensus.cc:606] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Received request to transfer leadership to TS 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:51.570906 21039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
mode: GRACEFUL
new_leader_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:57544
I20260709 13:18:51.571377 21039 raft_consensus.cc:606] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Received request to transfer leadership to TS 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:51.578580 20966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
mode: GRACEFUL
new_leader_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:60316
I20260709 13:18:51.579070 20966 raft_consensus.cc:606] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Received request to transfer leadership to TS 3106c79919c14b9b995f45270d557a1a
I20260709 13:18:51.604408 20966 raft_consensus.cc:3060] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:51.610688 21087 consensus_queue.cc:1048] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.616954 21113 raft_consensus.cc:993] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a: : Instructing follower 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:51.617434 21135 raft_consensus.cc:1081] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Signalling peer 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:51.619076 20965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:60308
I20260709 13:18:51.619621 20965 raft_consensus.cc:493] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:51.619975 20965 raft_consensus.cc:3060] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.623519 21113 consensus_queue.cc:1048] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.626613 20965 raft_consensus.cc:515] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.630117 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:51.632944 20965 leader_election.cc:290] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.634166 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f1e56bf4cc5416ea22348bc387b57a9" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:51.634780 21039 raft_consensus.cc:3055] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.635105 21039 raft_consensus.cc:740] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:51.635888 21039 consensus_queue.cc:260] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.637213 21039 raft_consensus.cc:3060] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.642340 21039 raft_consensus.cc:2468] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 2.
I20260709 13:18:51.643196 20929 leader_election.cc:304] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:51.643918 21086 raft_consensus.cc:2804] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.644340 21086 raft_consensus.cc:697] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:51.645076 21086 consensus_queue.cc:237] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.651466 20788 catalog_manager.cc:5697] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 1 to 2, leader changed from 3106c79919c14b9b995f45270d557a1a (127.10.4.67) to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:51.737527 21164 raft_consensus.cc:993] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578: : Instructing follower 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:51.737972 21129 raft_consensus.cc:1081] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Signalling peer 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:51.739449 20891 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a8e2647fab8b4079861f111406365333"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:60854
I20260709 13:18:51.739959 20891 raft_consensus.cc:493] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:51.740326 20891 raft_consensus.cc:3060] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.746556 20891 raft_consensus.cc:515] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.748286 20891 leader_election.cc:290] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.749233 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8e2647fab8b4079861f111406365333" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:51.749425 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8e2647fab8b4079861f111406365333" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:51.750038 20965 raft_consensus.cc:3055] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.750087 21039 raft_consensus.cc:3060] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.750341 20965 raft_consensus.cc:740] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:51.751184 20965 consensus_queue.cc:260] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.752279 20965 raft_consensus.cc:3060] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.755831 21039 raft_consensus.cc:2468] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 2.
I20260709 13:18:51.757113 20854 leader_election.cc:304] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:51.757861 21088 raft_consensus.cc:2804] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.757886 20965 raft_consensus.cc:2468] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 2.
I20260709 13:18:51.758324 21088 raft_consensus.cc:697] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:51.759240 21088 consensus_queue.cc:237] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.765950 20788 catalog_manager.cc:5697] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 1 to 2, leader changed from 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66) to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:51.787607 21106 raft_consensus.cc:993] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839: : Instructing follower 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:51.788075 21088 raft_consensus.cc:1081] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Signalling peer 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:51.789762 20965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:60292
I20260709 13:18:51.790280 20965 raft_consensus.cc:493] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:51.790660 20965 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.796013 20965 raft_consensus.cc:515] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.797575 20965 leader_election.cc:290] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.798820 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:51.799268 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:51.799566 21039 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.799916 20891 raft_consensus.cc:3055] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.800233 20891 raft_consensus.cc:740] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:51.801010 20891 consensus_queue.cc:260] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.802332 20891 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.806629 21039 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 2.
I20260709 13:18:51.807554 21129 consensus_queue.cc:1048] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:51.809504 20929 leader_election.cc:304] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:51.810525 21129 raft_consensus.cc:2804] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.810591 20891 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 2.
I20260709 13:18:51.811075 21129 raft_consensus.cc:697] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:51.812088 21129 consensus_queue.cc:237] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.816574 21086 raft_consensus.cc:993] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578: : Instructing follower 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:51.817082 21086 raft_consensus.cc:1081] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Signalling peer 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:51.820253 21039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:57530
I20260709 13:18:51.821421 21039 raft_consensus.cc:493] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:51.821607 21096 consensus_queue.cc:1048] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:51.821944 21039 raft_consensus.cc:3060] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.826216 20786 catalog_manager.cc:5697] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 1 to 2, leader changed from 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65) to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:51.829811 21039 raft_consensus.cc:515] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.832105 21039 leader_election.cc:290] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 2 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:51.835350 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:51.835752 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e6552f0674e9698832dfbd218b118" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:51.835995 20965 raft_consensus.cc:3055] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.836483 20965 raft_consensus.cc:740] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:51.837365 20965 consensus_queue.cc:260] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.838498 20965 raft_consensus.cc:3060] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.846422 20965 raft_consensus.cc:2468] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 2.
I20260709 13:18:51.847620 21087 raft_consensus.cc:993] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a: : Instructing follower 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:51.847779 21004 leader_election.cc:304] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:51.848165 21113 raft_consensus.cc:1081] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Signalling peer 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:51.849025 21087 raft_consensus.cc:2804] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.849480 21087 raft_consensus.cc:697] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 2 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:51.850000 20890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:60864
I20260709 13:18:51.850656 20890 raft_consensus.cc:493] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:51.850276 21087 consensus_queue.cc:237] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.852310 20890 raft_consensus.cc:3060] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.858350 20788 catalog_manager.cc:5697] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 1 to 2, leader changed from 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66) to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 2 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.859077 20890 raft_consensus.cc:515] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.861580 20890 leader_election.cc:290] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:51.861799 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:51.862521 20965 raft_consensus.cc:3060] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.862186 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa44f2202b74d6fa6853bff5d64d1d2" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:51.863040 21039 raft_consensus.cc:3055] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:51.863287 21039 raft_consensus.cc:740] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:51.864069 21039 consensus_queue.cc:260] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.865190 21039 raft_consensus.cc:3060] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:51.868834 20965 raft_consensus.cc:2468] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 2.
I20260709 13:18:51.869854 20855 leader_election.cc:304] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:51.870537 21088 raft_consensus.cc:2804] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:51.870586 21039 raft_consensus.cc:2468] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 2.
I20260709 13:18:51.870965 21088 raft_consensus.cc:697] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:51.871838 21088 consensus_queue.cc:237] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:51.877976 20788 catalog_manager.cc:5697] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 1 to 2, leader changed from 3106c79919c14b9b995f45270d557a1a (127.10.4.67) to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:52.028585 21135 raft_consensus.cc:993] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a: : Instructing follower 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:52.029057 21087 raft_consensus.cc:1081] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Signalling peer 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:52.030470 20890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:60864
I20260709 13:18:52.031036 20890 raft_consensus.cc:493] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:52.031392 20890 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.036841 20890 raft_consensus.cc:515] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:52.038353 20890 leader_election.cc:290] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:52.039352 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:52.039486 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:52.039996 21039 raft_consensus.cc:3055] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:52.040060 20965 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.040285 21039 raft_consensus.cc:740] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:52.041065 21039 consensus_queue.cc:260] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:52.041994 21039 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.045169 20965 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 2.
I20260709 13:18:52.046321 20855 leader_election.cc:304] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:52.046993 21088 raft_consensus.cc:2804] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:52.047138 21039 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 2.
I20260709 13:18:52.047386 21088 raft_consensus.cc:697] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:52.048317 21088 consensus_queue.cc:237] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:52.054505 20788 catalog_manager.cc:5697] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 1 to 2, leader changed from 3106c79919c14b9b995f45270d557a1a (127.10.4.67) to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:52.086814 21039 raft_consensus.cc:1275] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.088064 21129 consensus_queue.cc:1048] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.096052 20890 raft_consensus.cc:3060] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.104807 20890 raft_consensus.cc:1275] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.106163 21164 consensus_queue.cc:1048] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.140259 20965 raft_consensus.cc:1275] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.141614 21088 consensus_queue.cc:1048] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.149724 21039 raft_consensus.cc:1275] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.151731 21109 consensus_queue.cc:1048] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.247696 20890 raft_consensus.cc:3060] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:52.253449 20890 raft_consensus.cc:1275] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.254858 21087 consensus_queue.cc:1048] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.263988 20965 raft_consensus.cc:1275] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.265482 21135 consensus_queue.cc:1048] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.271755 21039 raft_consensus.cc:1275] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.272977 21164 consensus_queue.cc:1048] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.282992 20890 raft_consensus.cc:1275] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.284960 21086 consensus_queue.cc:1048] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.324765 21039 raft_consensus.cc:1275] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.325955 21109 consensus_queue.cc:1048] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.334380 20965 raft_consensus.cc:1275] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.335804 21106 consensus_queue.cc:1048] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.598667 20965 raft_consensus.cc:1275] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.599924 21106 consensus_queue.cc:1048] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.609268 21039 raft_consensus.cc:1275] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:52.610759 21109 consensus_queue.cc:1048] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:54.609177 20788 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49158:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:54.612047 20788 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:54.666589 20879 tablet_service.cc:1511] Processing CreateTablet for tablet 986b76c2b9d545afa7c5f88eca7c5346 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:54.667917 20949 tablet_service.cc:1511] Processing CreateTablet for tablet 986b76c2b9d545afa7c5f88eca7c5346 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:54.668521 20879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986b76c2b9d545afa7c5f88eca7c5346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.669127 21030 tablet_service.cc:1511] Processing CreateTablet for tablet 986b76c2b9d545afa7c5f88eca7c5346 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:54.669591 20949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986b76c2b9d545afa7c5f88eca7c5346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.670722 21030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986b76c2b9d545afa7c5f88eca7c5346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.672511 20951 tablet_service.cc:1511] Processing CreateTablet for tablet c9682d66d5a14a50a243c111a280b42c (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:54.673362 20878 tablet_service.cc:1511] Processing CreateTablet for tablet c9682d66d5a14a50a243c111a280b42c (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:54.673836 21027 tablet_service.cc:1511] Processing CreateTablet for tablet c9682d66d5a14a50a243c111a280b42c (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:54.673943 20951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9682d66d5a14a50a243c111a280b42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.674765 20878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9682d66d5a14a50a243c111a280b42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.675249 21027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9682d66d5a14a50a243c111a280b42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.686163 20876 tablet_service.cc:1511] Processing CreateTablet for tablet 62432463b07e4593be55c65c1ae972a7 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:54.687697 20876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62432463b07e4593be55c65c1ae972a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.687561 20955 tablet_service.cc:1511] Processing CreateTablet for tablet 62432463b07e4593be55c65c1ae972a7 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:54.689085 20955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62432463b07e4593be55c65c1ae972a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.714088 21029 tablet_service.cc:1511] Processing CreateTablet for tablet 62432463b07e4593be55c65c1ae972a7 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:54.715588 21029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62432463b07e4593be55c65c1ae972a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.727438 20874 tablet_service.cc:1511] Processing CreateTablet for tablet a60a556fcad84768926201542f32dd63 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:54.729151 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60a556fcad84768926201542f32dd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.734727 20949 tablet_service.cc:1511] Processing CreateTablet for tablet a60a556fcad84768926201542f32dd63 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:54.736158 21028 tablet_service.cc:1511] Processing CreateTablet for tablet a60a556fcad84768926201542f32dd63 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:54.737529 21028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60a556fcad84768926201542f32dd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.738225 20880 tablet_service.cc:1511] Processing CreateTablet for tablet 43fa8704ba3c47c6a962f4b9a36b8c74 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:54.739630 20880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fa8704ba3c47c6a962f4b9a36b8c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.740183 20950 tablet_service.cc:1511] Processing CreateTablet for tablet 43fa8704ba3c47c6a962f4b9a36b8c74 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:54.741261 21025 tablet_service.cc:1511] Processing CreateTablet for tablet 43fa8704ba3c47c6a962f4b9a36b8c74 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:54.742722 21025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fa8704ba3c47c6a962f4b9a36b8c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.736152 20949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60a556fcad84768926201542f32dd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.745004 20950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43fa8704ba3c47c6a962f4b9a36b8c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.748693 21186 tablet_bootstrap.cc:492] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:54.767279 20881 tablet_service.cc:1511] Processing CreateTablet for tablet 7835def769884fc69be515fa1c172ae8 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:54.768792 20881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7835def769884fc69be515fa1c172ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.774962 21186 tablet_bootstrap.cc:654] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.777326 20955 tablet_service.cc:1511] Processing CreateTablet for tablet 7835def769884fc69be515fa1c172ae8 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:54.790405 21028 tablet_service.cc:1511] Processing CreateTablet for tablet 7835def769884fc69be515fa1c172ae8 (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:54.792579 21187 tablet_bootstrap.cc:492] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:54.795444 21028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7835def769884fc69be515fa1c172ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.805603 20955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7835def769884fc69be515fa1c172ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.809597 20876 tablet_service.cc:1511] Processing CreateTablet for tablet e4e58e63b66b44b38d012f2e83ee8b3d (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:54.811074 20876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e58e63b66b44b38d012f2e83ee8b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.812734 21186 tablet_bootstrap.cc:492] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:54.813439 21186 ts_tablet_manager.cc:1403] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.005s
I20260709 13:18:54.815886 21187 tablet_bootstrap.cc:654] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.818216 21186 raft_consensus.cc:359] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.825758 21186 raft_consensus.cc:385] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.826206 21186 raft_consensus.cc:740] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.827692 21025 tablet_service.cc:1511] Processing CreateTablet for tablet e4e58e63b66b44b38d012f2e83ee8b3d (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:54.828020 21186 consensus_queue.cc:260] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.831087 20878 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1968ddaebf4232a8e0df881ee33b9f (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:54.832307 20878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1968ddaebf4232a8e0df881ee33b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.833549 21186 ts_tablet_manager.cc:1434] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.006s
I20260709 13:18:54.824985 21189 tablet_bootstrap.cc:492] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:54.834582 21186 tablet_bootstrap.cc:492] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:54.823161 20950 tablet_service.cc:1511] Processing CreateTablet for tablet e4e58e63b66b44b38d012f2e83ee8b3d (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:54.835587 20950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e58e63b66b44b38d012f2e83ee8b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.837190 21027 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1968ddaebf4232a8e0df881ee33b9f (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:54.829974 20949 tablet_service.cc:1511] Processing CreateTablet for tablet 5e1968ddaebf4232a8e0df881ee33b9f (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:54.829216 21025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e58e63b66b44b38d012f2e83ee8b3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.838959 20949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1968ddaebf4232a8e0df881ee33b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.844455 21189 tablet_bootstrap.cc:654] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.850154 21027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e1968ddaebf4232a8e0df881ee33b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.865084 21186 tablet_bootstrap.cc:654] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.866403 21189 tablet_bootstrap.cc:492] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:54.867018 21189 ts_tablet_manager.cc:1403] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.007s
I20260709 13:18:54.868918 20874 tablet_service.cc:1511] Processing CreateTablet for tablet 07e7f7268a2747c38dd024380d8dc95e (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:54.872298 20874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e7f7268a2747c38dd024380d8dc95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.878609 20950 tablet_service.cc:1511] Processing CreateTablet for tablet 07e7f7268a2747c38dd024380d8dc95e (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
W20260709 13:18:54.879223 20837 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:54.880352 20950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e7f7268a2747c38dd024380d8dc95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.882205 21025 tablet_service.cc:1511] Processing CreateTablet for tablet 07e7f7268a2747c38dd024380d8dc95e (DEFAULT_TABLE table=soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:54.883518 21025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07e7f7268a2747c38dd024380d8dc95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:54.887080 21187 tablet_bootstrap.cc:492] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:54.887698 21187 ts_tablet_manager.cc:1403] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.096s	user 0.019s	sys 0.026s
I20260709 13:18:54.870131 21189 raft_consensus.cc:359] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.890820 21187 raft_consensus.cc:359] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.891746 21187 raft_consensus.cc:385] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.892123 21187 raft_consensus.cc:740] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.892908 21187 consensus_queue.cc:260] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.891621 21189 raft_consensus.cc:385] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.899595 21186 tablet_bootstrap.cc:492] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:54.900164 21186 ts_tablet_manager.cc:1403] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.004s
I20260709 13:18:54.902199 21187 ts_tablet_manager.cc:1434] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:18:54.899667 21189 raft_consensus.cc:740] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.903213 21187 tablet_bootstrap.cc:492] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:54.903563 21189 consensus_queue.cc:260] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.908321 21189 ts_tablet_manager.cc:1434] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:18:54.909512 21186 raft_consensus.cc:359] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.910275 21186 raft_consensus.cc:385] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.910547 21186 raft_consensus.cc:740] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.910609 21187 tablet_bootstrap.cc:654] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.911949 21189 tablet_bootstrap.cc:492] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:54.917028 21186 consensus_queue.cc:260] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.918351 21189 tablet_bootstrap.cc:654] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.919607 21186 ts_tablet_manager.cc:1434] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 13:18:54.920723 21186 tablet_bootstrap.cc:492] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:54.927570 21186 tablet_bootstrap.cc:654] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.933001 21189 tablet_bootstrap.cc:492] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:54.933499 21189 ts_tablet_manager.cc:1403] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.006s
I20260709 13:18:54.933537 21186 tablet_bootstrap.cc:492] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:54.934157 21186 ts_tablet_manager.cc:1403] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.005s
I20260709 13:18:54.936295 21189 raft_consensus.cc:359] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.936867 21189 raft_consensus.cc:385] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.937021 21187 tablet_bootstrap.cc:492] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:54.937251 21189 raft_consensus.cc:740] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.937115 21186 raft_consensus.cc:359] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.937819 21187 ts_tablet_manager.cc:1403] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.035s	user 0.005s	sys 0.013s
I20260709 13:18:54.938140 21186 raft_consensus.cc:385] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.938453 21186 raft_consensus.cc:740] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.938421 21189 consensus_queue.cc:260] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.939356 21186 consensus_queue.cc:260] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.941155 21189 ts_tablet_manager.cc:1434] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 13:18:54.941025 21187 raft_consensus.cc:359] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.941751 21186 ts_tablet_manager.cc:1434] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:18:54.941828 21187 raft_consensus.cc:385] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.942200 21189 tablet_bootstrap.cc:492] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:54.942783 21186 tablet_bootstrap.cc:492] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:54.943650 21109 raft_consensus.cc:493] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.944144 21109 raft_consensus.cc:515] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.946388 21109 leader_election.cc:290] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:54.942231 21187 raft_consensus.cc:740] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.947679 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986b76c2b9d545afa7c5f88eca7c5346" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:54.948033 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986b76c2b9d545afa7c5f88eca7c5346" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:54.948170 21187 consensus_queue.cc:260] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.948596 21039 raft_consensus.cc:2468] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:54.949232 20965 raft_consensus.cc:2468] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:54.950230 20854 leader_election.cc:304] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:54.952397 21186 tablet_bootstrap.cc:654] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.953552 21109 raft_consensus.cc:2804] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:54.953953 21109 raft_consensus.cc:493] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.954315 21109 raft_consensus.cc:3060] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:54.957382 21187 ts_tablet_manager.cc:1434] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 13:18:54.958422 21187 tablet_bootstrap.cc:492] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:54.965099 21187 tablet_bootstrap.cc:654] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:54.966696 21109 raft_consensus.cc:515] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.970741 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986b76c2b9d545afa7c5f88eca7c5346" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:54.971326 20965 raft_consensus.cc:3060] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:54.971983 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986b76c2b9d545afa7c5f88eca7c5346" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:54.972713 21040 raft_consensus.cc:3060] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:54.972831 21189 tablet_bootstrap.cc:654] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:54.971199 21066 tablet.cc:2406] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:18:54.982983 21187 tablet_bootstrap.cc:492] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:54.983778 21187 ts_tablet_manager.cc:1403] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.005s
I20260709 13:18:54.986673 21040 raft_consensus.cc:2468] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:54.988358 20854 leader_election.cc:304] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:54.989395 21185 raft_consensus.cc:2804] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:54.992656 20965 raft_consensus.cc:2468] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:54.994231 21187 raft_consensus.cc:359] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:54.995102 21187 raft_consensus.cc:385] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:54.995424 21187 raft_consensus.cc:740] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:54.995932 21109 leader_election.cc:290] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:54.996241 21187 consensus_queue.cc:260] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.001624 21185 raft_consensus.cc:697] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.002817 21185 consensus_queue.cc:237] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.009232 21186 tablet_bootstrap.cc:492] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:55.009929 21186 ts_tablet_manager.cc:1403] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.047s
I20260709 13:18:55.008226 21187 ts_tablet_manager.cc:1434] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:18:55.010896 21189 tablet_bootstrap.cc:492] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:55.011441 21189 ts_tablet_manager.cc:1403] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.069s	user 0.027s	sys 0.015s
I20260709 13:18:55.013980 21187 tablet_bootstrap.cc:492] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:55.013573 21186 raft_consensus.cc:359] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.014532 21186 raft_consensus.cc:385] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.014900 21186 raft_consensus.cc:740] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.015672 21186 consensus_queue.cc:260] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.018180 21186 ts_tablet_manager.cc:1434] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:55.019325 21186 tablet_bootstrap.cc:492] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:55.019311 21189 raft_consensus.cc:359] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.020218 21189 raft_consensus.cc:385] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.020519 21189 raft_consensus.cc:740] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.021492 21189 consensus_queue.cc:260] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.025362 21189 ts_tablet_manager.cc:1434] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:18:55.026305 21189 tablet_bootstrap.cc:492] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:55.028175 21186 tablet_bootstrap.cc:654] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.029307 21199 raft_consensus.cc:493] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.029830 21199 raft_consensus.cc:515] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.032099 21199 leader_election.cc:290] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.032775 21189 tablet_bootstrap.cc:654] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.038358 21187 tablet_bootstrap.cc:654] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.039475 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9682d66d5a14a50a243c111a280b42c" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:55.040162 21040 raft_consensus.cc:2468] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.040949 21186 tablet_bootstrap.cc:492] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:55.041496 21186 ts_tablet_manager.cc:1403] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 13:18:55.041697 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9682d66d5a14a50a243c111a280b42c" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:55.042320 20965 raft_consensus.cc:2468] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.042982 21189 tablet_bootstrap.cc:492] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:55.043603 21189 ts_tablet_manager.cc:1403] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:18:55.043780 20854 leader_election.cc:304] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:55.044562 21186 raft_consensus.cc:359] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.045377 21186 raft_consensus.cc:385] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.045590 21109 raft_consensus.cc:2804] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.045650 21186 raft_consensus.cc:740] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.046096 21109 raft_consensus.cc:493] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.046437 21109 raft_consensus.cc:3060] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.046613 21186 consensus_queue.cc:260] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.046945 21189 raft_consensus.cc:359] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.047775 21189 raft_consensus.cc:385] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.048074 21189 raft_consensus.cc:740] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.049188 21186 ts_tablet_manager.cc:1434] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.003s
I20260709 13:18:55.048827 21189 consensus_queue.cc:260] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.050308 21186 tablet_bootstrap.cc:492] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:55.051260 21189 ts_tablet_manager.cc:1434] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:18:55.052424 21189 tablet_bootstrap.cc:492] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:55.057492 21186 tablet_bootstrap.cc:654] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.059167 21189 tablet_bootstrap.cc:654] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.062201 21109 raft_consensus.cc:515] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.070206 21186 tablet_bootstrap.cc:492] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:55.070199 21187 tablet_bootstrap.cc:492] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:55.070868 21189 tablet_bootstrap.cc:492] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:55.070972 21186 ts_tablet_manager.cc:1403] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:18:55.071069 21187 ts_tablet_manager.cc:1403] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.011s
I20260709 13:18:55.071480 21189 ts_tablet_manager.cc:1403] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.010s
I20260709 13:18:55.074229 21186 raft_consensus.cc:359] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.074502 21187 raft_consensus.cc:359] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.074990 21186 raft_consensus.cc:385] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.074802 21189 raft_consensus.cc:359] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.075276 21187 raft_consensus.cc:385] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.075434 21186 raft_consensus.cc:740] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.075579 21189 raft_consensus.cc:385] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.075794 21187 raft_consensus.cc:740] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.076046 21189 raft_consensus.cc:740] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.076388 21186 consensus_queue.cc:260] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.076977 21189 consensus_queue.cc:260] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.078820 21186 ts_tablet_manager.cc:1434] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:55.079355 21189 ts_tablet_manager.cc:1434] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:18:55.079794 21186 tablet_bootstrap.cc:492] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:55.080075 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9682d66d5a14a50a243c111a280b42c" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:55.080425 21189 tablet_bootstrap.cc:492] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:55.080627 20965 raft_consensus.cc:3060] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.082304 20787 catalog_manager.cc:5697] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.086781 21186 tablet_bootstrap.cc:654] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.087430 21189 tablet_bootstrap.cc:654] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.088652 20965 raft_consensus.cc:2468] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.090114 20855 leader_election.cc:304] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.091223 21185 raft_consensus.cc:2804] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.091846 21185 raft_consensus.cc:697] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.092945 21185 consensus_queue.cc:237] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.076701 21187 consensus_queue.cc:260] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.094712 21135 raft_consensus.cc:493] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.096904 21135 raft_consensus.cc:515] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.099243 21186 tablet_bootstrap.cc:492] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:55.095804 21109 leader_election.cc:290] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.099812 21186 ts_tablet_manager.cc:1403] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.002s
I20260709 13:18:55.108414 21186 raft_consensus.cc:359] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.109757 21186 raft_consensus.cc:385] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:18:55.107136 20854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:35839, user_credentials={real_user=slave}} blocked reactor thread for 55779.4us
I20260709 13:18:55.110929 21186 raft_consensus.cc:740] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.111850 21135 leader_election.cc:290] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:55.114387 21187 ts_tablet_manager.cc:1434] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.003s
I20260709 13:18:55.114710 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9682d66d5a14a50a243c111a280b42c" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.115024 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60a556fcad84768926201542f32dd63" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:55.115847 21040 raft_consensus.cc:3060] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.116282 20965 raft_consensus.cc:2468] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 0.
I20260709 13:18:55.117285 21189 tablet_bootstrap.cc:492] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:55.117909 21189 ts_tablet_manager.cc:1403] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.008s
I20260709 13:18:55.117844 21004 leader_election.cc:304] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.119093 21135 raft_consensus.cc:2804] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.119521 21135 raft_consensus.cc:493] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.119918 21135 raft_consensus.cc:3060] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.121286 21189 raft_consensus.cc:359] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.112124 21186 consensus_queue.cc:260] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.122148 21189 raft_consensus.cc:385] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.122474 21189 raft_consensus.cc:740] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.123189 21189 consensus_queue.cc:260] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.124640 21040 raft_consensus.cc:2468] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.122148 21187 tablet_bootstrap.cc:492] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:55.125551 21189 ts_tablet_manager.cc:1434] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:55.126996 21189 tablet_bootstrap.cc:492] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:55.134063 21187 tablet_bootstrap.cc:654] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.136806 20890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60a556fcad84768926201542f32dd63" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:55.137495 20890 raft_consensus.cc:2468] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 0.
I20260709 13:18:55.141233 21211 raft_consensus.cc:493] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.141893 21211 raft_consensus.cc:515] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.147600 21211 leader_election.cc:290] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:55.148497 21186 ts_tablet_manager.cc:1434] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.008s
I20260709 13:18:55.148578 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
W20260709 13:18:55.156504 21004 leader_election.cc:343] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:55.156877 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:55.157547 21186 tablet_bootstrap.cc:492] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
W20260709 13:18:55.158463 21006 leader_election.cc:343] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:55.158936 21006 leader_election.cc:304] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a; no voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:55.160209 21213 raft_consensus.cc:2749] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:55.161362 20788 catalog_manager.cc:5697] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.160051 21135 raft_consensus.cc:515] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.162308 21189 tablet_bootstrap.cc:654] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.164781 21187 tablet_bootstrap.cc:492] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:55.168962 21135 leader_election.cc:290] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:55.170526 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60a556fcad84768926201542f32dd63" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:55.171711 21187 ts_tablet_manager.cc:1403] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.007s
I20260709 13:18:55.172258 20891 raft_consensus.cc:3060] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.172444 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60a556fcad84768926201542f32dd63" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:55.173055 20965 raft_consensus.cc:3060] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.174919 21189 tablet_bootstrap.cc:492] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:55.175521 21189 ts_tablet_manager.cc:1403] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.007s
I20260709 13:18:55.175840 21187 raft_consensus.cc:359] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.177641 21187 raft_consensus.cc:385] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.178009 21187 raft_consensus.cc:740] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.178694 21189 raft_consensus.cc:359] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.178824 21187 consensus_queue.cc:260] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.179601 21189 raft_consensus.cc:385] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.179924 21189 raft_consensus.cc:740] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.180696 21189 consensus_queue.cc:260] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.181705 21187 ts_tablet_manager.cc:1434] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:18:55.182348 21186 tablet_bootstrap.cc:654] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.183486 21187 tablet_bootstrap.cc:492] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:55.184716 20965 raft_consensus.cc:2468] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 1.
I20260709 13:18:55.186239 21004 leader_election.cc:304] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.184716 20891 raft_consensus.cc:2468] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 1.
I20260709 13:18:55.191032 21187 tablet_bootstrap.cc:654] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.191880 21186 tablet_bootstrap.cc:492] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:55.192441 21186 ts_tablet_manager.cc:1403] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.005s
I20260709 13:18:55.195348 21186 raft_consensus.cc:359] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.196027 21186 raft_consensus.cc:385] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.196287 21186 raft_consensus.cc:740] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.196890 21186 consensus_queue.cc:260] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.198132 21189 ts_tablet_manager.cc:1434] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:18:55.199453 21189 tablet_bootstrap.cc:492] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:55.203624 21187 tablet_bootstrap.cc:492] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:55.204283 21187 ts_tablet_manager.cc:1403] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 13:18:55.205039 21135 raft_consensus.cc:2804] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.205727 21135 raft_consensus.cc:697] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:55.213936 21189 tablet_bootstrap.cc:654] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.214238 21187 raft_consensus.cc:359] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.218961 21185 raft_consensus.cc:493] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.220227 21187 raft_consensus.cc:385] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.220368 21185 raft_consensus.cc:515] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.220721 21187 raft_consensus.cc:740] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.221783 21187 consensus_queue.cc:260] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.222482 21185 leader_election.cc:290] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.223626 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7835def769884fc69be515fa1c172ae8" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:55.206557 21135 consensus_queue.cc:237] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.224390 20965 raft_consensus.cc:2468] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.223737 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7835def769884fc69be515fa1c172ae8" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:55.225520 21040 raft_consensus.cc:2468] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.226517 20855 leader_election.cc:304] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.227871 21185 raft_consensus.cc:2804] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.228688 21186 ts_tablet_manager.cc:1434] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.000s
I20260709 13:18:55.229856 21186 tablet_bootstrap.cc:492] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a: Bootstrap starting.
I20260709 13:18:55.234301 21187 ts_tablet_manager.cc:1434] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.005s
I20260709 13:18:55.228257 21185 raft_consensus.cc:493] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.237401 21185 raft_consensus.cc:3060] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.243321 21186 tablet_bootstrap.cc:654] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.245213 21187 tablet_bootstrap.cc:492] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:55.254200 21187 tablet_bootstrap.cc:654] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.255983 21204 raft_consensus.cc:493] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.256088 21186 tablet_bootstrap.cc:492] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a: No bootstrap required, opened a new log
I20260709 13:18:55.256888 21186 ts_tablet_manager.cc:1403] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.006s
I20260709 13:18:55.256661 21204 raft_consensus.cc:515] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.257184 21189 tablet_bootstrap.cc:492] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:55.257813 21189 ts_tablet_manager.cc:1403] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.059s	user 0.026s	sys 0.027s
I20260709 13:18:55.260638 21186 raft_consensus.cc:359] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.261201 21189 raft_consensus.cc:359] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.262204 21186 raft_consensus.cc:385] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.262285 21189 raft_consensus.cc:385] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.263638 21204 leader_election.cc:290] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.264042 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62432463b07e4593be55c65c1ae972a7" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:55.264941 21040 raft_consensus.cc:2468] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.266135 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62432463b07e4593be55c65c1ae972a7" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:55.266949 20965 raft_consensus.cc:2468] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.263113 21189 raft_consensus.cc:740] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.267336 20854 leader_election.cc:304] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:55.268039 21189 consensus_queue.cc:260] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.268677 21187 tablet_bootstrap.cc:492] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:55.269196 21199 raft_consensus.cc:2804] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.269241 21187 ts_tablet_manager.cc:1403] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.013s
I20260709 13:18:55.269701 21199 raft_consensus.cc:493] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.270051 21186 raft_consensus.cc:740] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.270164 21199 raft_consensus.cc:3060] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.270449 21189 ts_tablet_manager.cc:1434] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:18:55.270961 21186 consensus_queue.cc:260] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.271533 21189 tablet_bootstrap.cc:492] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839: Bootstrap starting.
I20260709 13:18:55.273751 21186 ts_tablet_manager.cc:1434] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:18:55.275907 21187 raft_consensus.cc:359] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.276863 21187 raft_consensus.cc:385] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.277196 21187 raft_consensus.cc:740] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.278213 21189 tablet_bootstrap.cc:654] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.278210 21187 consensus_queue.cc:260] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.280618 20787 catalog_manager.cc:5697] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 1 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:55.280966 21187 ts_tablet_manager.cc:1434] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:18:55.282959 21199 raft_consensus.cc:515] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.292011 21187 tablet_bootstrap.cc:492] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:55.286408 21189 tablet_bootstrap.cc:492] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839: No bootstrap required, opened a new log
I20260709 13:18:55.292742 21189 ts_tablet_manager.cc:1403] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 13:18:55.295606 21199 leader_election.cc:290] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.286131 21185 raft_consensus.cc:515] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.297137 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62432463b07e4593be55c65c1ae972a7" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.297812 21040 raft_consensus.cc:3060] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.298753 21185 leader_election.cc:290] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.300175 21187 tablet_bootstrap.cc:654] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.301767 20965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62432463b07e4593be55c65c1ae972a7" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:55.302369 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7835def769884fc69be515fa1c172ae8" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:55.303018 20966 raft_consensus.cc:3060] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.300772 21189 raft_consensus.cc:359] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.304565 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7835def769884fc69be515fa1c172ae8" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.304852 21189 raft_consensus.cc:385] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.305156 21039 raft_consensus.cc:3060] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.305186 21189 raft_consensus.cc:740] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.302379 20965 raft_consensus.cc:3060] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.311569 20966 raft_consensus.cc:2468] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.313172 20855 leader_election.cc:304] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.313482 21039 raft_consensus.cc:2468] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.314281 21185 raft_consensus.cc:2804] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.315742 21185 raft_consensus.cc:697] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.306821 21189 consensus_queue.cc:260] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.321753 21189 ts_tablet_manager.cc:1434] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:18:55.321132 21185 consensus_queue.cc:237] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.322577 20965 raft_consensus.cc:2468] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.324579 20855 leader_election.cc:304] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.325353 21199 raft_consensus.cc:2804] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.325834 21199 raft_consensus.cc:697] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.326805 21187 tablet_bootstrap.cc:492] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:55.326671 21199 consensus_queue.cc:237] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.327387 21187 ts_tablet_manager.cc:1403] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.012s
I20260709 13:18:55.326578 21040 raft_consensus.cc:2468] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.331316 21187 raft_consensus.cc:359] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.332166 21187 raft_consensus.cc:385] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.332422 21187 raft_consensus.cc:740] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.333104 21187 consensus_queue.cc:260] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.335060 21187 ts_tablet_manager.cc:1434] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:18:55.336284 21187 tablet_bootstrap.cc:492] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578: Bootstrap starting.
I20260709 13:18:55.340576 21187 tablet_bootstrap.cc:654] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:55.340030 20789 catalog_manager.cc:5697] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.345628 21187 tablet_bootstrap.cc:492] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578: No bootstrap required, opened a new log
I20260709 13:18:55.346065 21187 ts_tablet_manager.cc:1403] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:18:55.346441 21086 raft_consensus.cc:493] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.346943 21086 raft_consensus.cc:515] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.349125 21086 leader_election.cc:290] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.348457 21187 raft_consensus.cc:359] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.350333 21187 raft_consensus.cc:385] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:55.349864 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fa8704ba3c47c6a962f4b9a36b8c74" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:55.350608 21187 raft_consensus.cc:740] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Initialized, Role: FOLLOWER
I20260709 13:18:55.350865 20891 raft_consensus.cc:2468] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:55.351261 21187 consensus_queue.cc:260] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.353250 21187 ts_tablet_manager.cc:1434] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:55.353044 20932 leader_election.cc:304] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.354866 21184 raft_consensus.cc:2804] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.355046 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fa8704ba3c47c6a962f4b9a36b8c74" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:55.355242 21184 raft_consensus.cc:493] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.355654 21184 raft_consensus.cc:3060] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.355690 21040 raft_consensus.cc:2468] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:55.358834 20787 catalog_manager.cc:5697] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.362725 21184 raft_consensus.cc:515] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.365545 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fa8704ba3c47c6a962f4b9a36b8c74" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:55.365751 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43fa8704ba3c47c6a962f4b9a36b8c74" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.366109 20891 raft_consensus.cc:3060] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.366384 21040 raft_consensus.cc:3060] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.367605 21184 leader_election.cc:290] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.372574 20891 raft_consensus.cc:2468] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:55.373191 21040 raft_consensus.cc:2468] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:55.373637 20932 leader_election.cc:304] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.374404 21184 raft_consensus.cc:2804] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.374780 21184 raft_consensus.cc:697] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:55.375490 21184 consensus_queue.cc:237] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.382517 20789 catalog_manager.cc:5697] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.426086 21184 raft_consensus.cc:493] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.426712 21184 raft_consensus.cc:515] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.428393 21184 leader_election.cc:290] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.429174 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1968ddaebf4232a8e0df881ee33b9f" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:55.429320 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1968ddaebf4232a8e0df881ee33b9f" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:55.429833 20891 raft_consensus.cc:2468] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:55.429908 21040 raft_consensus.cc:2468] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:55.430823 20932 leader_election.cc:304] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.431540 21184 raft_consensus.cc:2804] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.431842 21184 raft_consensus.cc:493] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.432119 21184 raft_consensus.cc:3060] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.437458 21184 raft_consensus.cc:515] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.439009 21184 leader_election.cc:290] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.439733 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1968ddaebf4232a8e0df881ee33b9f" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:55.440040 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e1968ddaebf4232a8e0df881ee33b9f" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.440404 20891 raft_consensus.cc:3060] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.440587 21040 raft_consensus.cc:3060] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.447717 21040 raft_consensus.cc:2468] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:55.447703 20891 raft_consensus.cc:2468] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:55.449132 20929 leader_election.cc:304] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.449843 21184 raft_consensus.cc:2804] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.450269 21184 raft_consensus.cc:697] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:55.450881 21184 consensus_queue.cc:237] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.459218 20789 catalog_manager.cc:5697] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.473482 21184 raft_consensus.cc:493] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.474076 21184 raft_consensus.cc:515] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.475947 21184 leader_election.cc:290] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.476548 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839" is_pre_election: true
I20260709 13:18:55.476994 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:55.477211 20891 raft_consensus.cc:2468] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:55.477648 21040 raft_consensus.cc:2468] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 0.
I20260709 13:18:55.478492 20932 leader_election.cc:304] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.479310 21184 raft_consensus.cc:2804] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.479719 21184 raft_consensus.cc:493] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.480034 21184 raft_consensus.cc:3060] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.480058 21185 consensus_queue.cc:1048] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.486757 21184 raft_consensus.cc:515] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.490491 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.491122 21039 raft_consensus.cc:3060] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.491988 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:55.492785 20891 raft_consensus.cc:3060] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.498968 21204 consensus_queue.cc:1048] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:55.500651 20891 raft_consensus.cc:2468] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:55.502087 20932 leader_election.cc:304] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.503139 21086 raft_consensus.cc:2804] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.504084 21204 raft_consensus.cc:493] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.504639 21204 raft_consensus.cc:515] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.506436 21204 leader_election.cc:290] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.507176 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578" is_pre_election: true
I20260709 13:18:55.507913 20966 raft_consensus.cc:2468] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.508682 21038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a" is_pre_election: true
I20260709 13:18:55.509326 20855 leader_election.cc:304] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.509445 21038 raft_consensus.cc:2468] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 0.
I20260709 13:18:55.510902 21199 raft_consensus.cc:2804] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:55.513633 21199 raft_consensus.cc:493] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.514001 21199 raft_consensus.cc:3060] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.515594 21204 consensus_queue.cc:1048] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.518126 21086 raft_consensus.cc:697] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:55.518992 21039 raft_consensus.cc:2468] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 1.
I20260709 13:18:55.519174 21086 consensus_queue.cc:237] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.522490 21184 leader_election.cc:290] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 1 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.524305 21199 raft_consensus.cc:515] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.527446 21039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.527578 20964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:55.528338 21039 raft_consensus.cc:3060] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.528481 20964 raft_consensus.cc:3060] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:55.531615 21199 leader_election.cc:290] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.541719 20964 raft_consensus.cc:2468] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.542219 21204 consensus_queue.cc:1048] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:55.543704 20855 leader_election.cc:304] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.544570 21109 raft_consensus.cc:2804] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:55.545071 21109 raft_consensus.cc:697] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.545917 21109 consensus_queue.cc:237] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.550503 21039 raft_consensus.cc:2468] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 1.
I20260709 13:18:55.559635 20789 catalog_manager.cc:5697] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 1 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.564945 20787 catalog_manager.cc:5697] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 1 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.612067 21038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a60a556fcad84768926201542f32dd63"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
mode: GRACEFUL
new_leader_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:57544
I20260709 13:18:55.612638 21038 raft_consensus.cc:606] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Received request to transfer leadership to TS 5b97fb429b274b6087bc1de914d8f839
I20260709 13:18:55.620847 20891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7835def769884fc69be515fa1c172ae8"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
mode: GRACEFUL
new_leader_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:60880
I20260709 13:18:55.621384 20891 raft_consensus.cc:606] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Received request to transfer leadership to TS 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:55.625801 20891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
mode: GRACEFUL
new_leader_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:60880
I20260709 13:18:55.626312 20891 raft_consensus.cc:606] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Received request to transfer leadership to TS 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:55.633792 20965 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
mode: GRACEFUL
new_leader_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:60316
I20260709 13:18:55.634258 20965 raft_consensus.cc:606] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Received request to transfer leadership to TS 3106c79919c14b9b995f45270d557a1a
I20260709 13:18:55.710259 21109 consensus_queue.cc:1048] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.716908 21135 consensus_queue.cc:1048] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.725989 21185 consensus_queue.cc:1048] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.730072 21240 raft_consensus.cc:993] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: : Instructing follower 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:55.730633 21135 raft_consensus.cc:1081] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Signalling peer 5b97fb429b274b6087bc1de914d8f839 to start an election
I20260709 13:18:55.734254 20890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a60a556fcad84768926201542f32dd63"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
 from {username='slave'} at 127.0.0.1:60864
I20260709 13:18:55.735875 20890 raft_consensus.cc:493] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:55.736339 20890 raft_consensus.cc:3060] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.735585 21135 consensus_queue.cc:1048] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:55.746603 20890 raft_consensus.cc:515] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.752800 20890 leader_election.cc:290] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Requested vote from peers 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.753011 21040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60a556fcad84768926201542f32dd63" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.753683 21040 raft_consensus.cc:3055] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:55.753974 21040 raft_consensus.cc:740] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:55.754812 21040 consensus_queue.cc:260] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.754724 20964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60a556fcad84768926201542f32dd63" candidate_uuid: "5b97fb429b274b6087bc1de914d8f839" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:55.756038 21040 raft_consensus.cc:3060] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.763813 21040 raft_consensus.cc:2468] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b97fb429b274b6087bc1de914d8f839 in term 2.
I20260709 13:18:55.765276 20854 leader_election.cc:304] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 977832f370f745c7a0cb21e3dca7d578
I20260709 13:18:55.766825 21185 raft_consensus.cc:2804] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:55.767583 21185 raft_consensus.cc:697] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Becoming Leader. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.768433 21185 consensus_queue.cc:237] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.778203 20786 catalog_manager.cc:5697] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 reported cstate change: term changed from 1 to 2, leader changed from 3106c79919c14b9b995f45270d557a1a (127.10.4.67) to 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65). New cstate: current_term: 2 leader_uuid: "5b97fb429b274b6087bc1de914d8f839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.786605 21185 consensus_queue.cc:1048] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.791381 21086 consensus_queue.cc:1048] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.797784 21185 consensus_queue.cc:1048] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.805024 21185 raft_consensus.cc:993] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: : Instructing follower 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:55.805441 21204 raft_consensus.cc:1081] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Signalling peer 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:55.811702 21184 consensus_queue.cc:1048] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:55.813247 20965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7835def769884fc69be515fa1c172ae8"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:60292
I20260709 13:18:55.813862 20965 raft_consensus.cc:493] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:55.814213 20965 raft_consensus.cc:3060] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.820627 20965 raft_consensus.cc:515] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.823295 20965 leader_election.cc:290] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.824877 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7835def769884fc69be515fa1c172ae8" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:55.825551 20891 raft_consensus.cc:3055] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:55.825855 20891 raft_consensus.cc:740] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.826615 20891 consensus_queue.cc:260] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.827910 20891 raft_consensus.cc:3060] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.824390 21038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7835def769884fc69be515fa1c172ae8" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.828940 21038 raft_consensus.cc:3060] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.836228 20891 raft_consensus.cc:2468] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 2.
I20260709 13:18:55.837451 20932 leader_election.cc:304] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b97fb429b274b6087bc1de914d8f839, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.838115 21232 raft_consensus.cc:2804] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:55.838522 21232 raft_consensus.cc:697] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:55.839335 21232 consensus_queue.cc:237] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.840994 21038 raft_consensus.cc:2468] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 2.
I20260709 13:18:55.845353 21232 consensus_queue.cc:1048] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.846403 20786 catalog_manager.cc:5697] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 1 to 2, leader changed from 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65) to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.856840 21086 consensus_queue.cc:1048] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.927995 21109 consensus_queue.cc:1048] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.934013 21185 raft_consensus.cc:993] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: : Instructing follower 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:55.934456 21199 raft_consensus.cc:1081] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Signalling peer 977832f370f745c7a0cb21e3dca7d578 to start an election
I20260709 13:18:55.936134 20964 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
 from {username='slave'} at 127.0.0.1:60292
I20260709 13:18:55.936712 20964 raft_consensus.cc:493] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:55.937103 20964 raft_consensus.cc:3060] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.940351 21199 consensus_queue.cc:1048] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.944345 20964 raft_consensus.cc:515] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.947937 21038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3106c79919c14b9b995f45270d557a1a"
I20260709 13:18:55.947947 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e58e63b66b44b38d012f2e83ee8b3d" candidate_uuid: "977832f370f745c7a0cb21e3dca7d578" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:55.948546 21038 raft_consensus.cc:3060] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.948926 20891 raft_consensus.cc:3055] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:55.949251 20891 raft_consensus.cc:740] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:55.950024 20891 consensus_queue.cc:260] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.951115 20891 raft_consensus.cc:3060] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:55.948040 20964 leader_election.cc:290] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 3106c79919c14b9b995f45270d557a1a (127.10.4.67:35839)
I20260709 13:18:55.956261 21038 raft_consensus.cc:2468] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 2.
I20260709 13:18:55.957574 20929 leader_election.cc:304] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:55.957937 20891 raft_consensus.cc:2468] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977832f370f745c7a0cb21e3dca7d578 in term 2.
I20260709 13:18:55.958355 21086 raft_consensus.cc:2804] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:55.958848 21086 raft_consensus.cc:697] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Becoming Leader. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:55.959472 21086 consensus_queue.cc:237] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:55.965906 20786 catalog_manager.cc:5697] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 reported cstate change: term changed from 1 to 2, leader changed from 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65) to 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66). New cstate: current_term: 2 leader_uuid: "977832f370f745c7a0cb21e3dca7d578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:55.985508 21184 consensus_queue.cc:1048] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:55.996577 21184 consensus_queue.cc:1048] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:56.005790 21184 raft_consensus.cc:993] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: : Instructing follower 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:56.006259 21086 raft_consensus.cc:1081] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Signalling peer 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:56.007598 21038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:57530
I20260709 13:18:56.008136 21038 raft_consensus.cc:493] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:56.008517 21038 raft_consensus.cc:3060] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:56.014324 21038 raft_consensus.cc:515] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:56.015853 21038 leader_election.cc:290] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 2 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:56.016510 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:56.016929 20964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07e7f7268a2747c38dd024380d8dc95e" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:56.017104 20891 raft_consensus.cc:3060] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:56.017452 20964 raft_consensus.cc:3055] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:56.017753 20964 raft_consensus.cc:740] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:56.018353 20964 consensus_queue.cc:260] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:56.019263 20964 raft_consensus.cc:3060] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:56.021991 20891 raft_consensus.cc:2468] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 2.
I20260709 13:18:56.022949 21006 leader_election.cc:304] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:56.023660 21242 raft_consensus.cc:2804] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:56.024124 21242 raft_consensus.cc:697] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 2 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:56.024902 21242 consensus_queue.cc:237] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:56.031939 20964 raft_consensus.cc:2468] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 2.
I20260709 13:18:56.033684 20786 catalog_manager.cc:5697] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 1 to 2, leader changed from 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66) to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 2 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:56.176190 21040 raft_consensus.cc:1275] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.177349 21204 consensus_queue.cc:1048] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.184156 20964 raft_consensus.cc:3060] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:56.191157 20964 raft_consensus.cc:1275] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Refusing update from remote peer 5b97fb429b274b6087bc1de914d8f839: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.192528 21199 consensus_queue.cc:1048] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.300377 20891 raft_consensus.cc:1275] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.301416 21086 consensus_queue.cc:1048] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.309463 21040 raft_consensus.cc:1275] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.310853 21250 consensus_queue.cc:1048] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.403731 20891 raft_consensus.cc:1275] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.404815 21242 consensus_queue.cc:1048] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.407795 20890 raft_consensus.cc:1275] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.409264 21250 consensus_queue.cc:1048] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.413486 20964 raft_consensus.cc:1275] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.415838 21242 consensus_queue.cc:1048] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.430985 21038 raft_consensus.cc:1275] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Refusing update from remote peer 977832f370f745c7a0cb21e3dca7d578: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:56.432950 21184 consensus_queue.cc:1048] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:58.637527 20786 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49158:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:18:58.658948 20786 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49158:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:18:58.692595 10257 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:58.866406 21299 tablet.cc:1724] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:58.867888 21302 tablet.cc:1724] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:58.872398 21301 tablet.cc:1724] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:58.917434 21290 log.cc:927] Time spent T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: Append to log took a long time: real 0.112s	user 0.003s	sys 0.000s
I20260709 13:18:58.956218 20889 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac"
dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
mode: GRACEFUL
new_leader_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:60890
I20260709 13:18:58.963984 20889 raft_consensus.cc:606] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Received request to transfer leadership to TS 3106c79919c14b9b995f45270d557a1a
W20260709 13:18:58.972159 21309 log.cc:927] Time spent T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:58.984575 21314 tablet.cc:1724] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:58.987538 20929 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.4.67:35839, user_credentials={real_user=slave}} blocked reactor thread for 84033.3us
I20260709 13:18:58.907783 21307 tablet.cc:1724] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.043512 21323 tablet.cc:1724] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:59.053164 21316 log.cc:927] Time spent T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:59.070420 21301 tablet.cc:1724] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.084671 21302 tablet.cc:1724] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.117722 21325 tablet.cc:1724] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.118201 21301 tablet.cc:1724] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.119874 21331 tablet.cc:1724] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.025288 21315 tablet.cc:1724] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:59.123096 21326 log.cc:927] Time spent T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:18:59.129146 21322 log.cc:927] Time spent T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:18:59.159513 21302 tablet.cc:1724] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.132586 21299 tablet.cc:1724] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.140046 21314 tablet.cc:1724] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.149940 21325 tablet.cc:1724] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.150305 21329 tablet.cc:1724] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.151163 21323 tablet.cc:1724] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:59.156754 21321 log.cc:927] Time spent T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:18:59.159636 21330 tablet.cc:1724] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.184082 21304 raft_consensus.cc:993] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839: : Instructing follower 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:59.184582 21304 raft_consensus.cc:1081] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Signalling peer 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:59.186990 20966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3"
dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
mode: GRACEFUL
new_leader_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:41356
I20260709 13:18:59.187664 20966 raft_consensus.cc:606] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Received request to transfer leadership to TS 3106c79919c14b9b995f45270d557a1a
I20260709 13:18:59.189409 10257 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:59.190064 10257 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:59.192667 21315 tablet.cc:1724] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.197621 21040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:57532
I20260709 13:18:59.198259 21040 raft_consensus.cc:493] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:59.198649 21040 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.203912 21329 tablet.cc:1724] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.206220 21314 tablet.cc:1724] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.208461 21040 raft_consensus.cc:515] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:59.139472 21324 tablet.cc:1724] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.211217 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:59.211854 20891 raft_consensus.cc:3055] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:59.212138 20891 raft_consensus.cc:740] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b97fb429b274b6087bc1de914d8f839, State: Running, Role: LEADER
I20260709 13:18:59.212859 20891 consensus_queue.cc:260] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:59.213912 20891 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.212944 21325 tablet.cc:1724] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.221078 21331 tablet.cc:1724] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.222605 21307 tablet.cc:1724] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.226222 21040 leader_election.cc:290] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 3 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:59.227368 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab942b20de44ae7bab51d2bf83142ac" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:59.232267 20966 raft_consensus.cc:3060] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.217860 21330 tablet.cc:1724] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:59.239419 20966 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 3.
I20260709 13:18:59.240082 20891 raft_consensus.cc:2468] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 3.
I20260709 13:18:59.241217 21004 leader_election.cc:304] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 977832f370f745c7a0cb21e3dca7d578; no voters: 
I20260709 13:18:59.242158 21240 raft_consensus.cc:2804] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:59.242602 21240 raft_consensus.cc:697] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 3 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:59.243463 21240 consensus_queue.cc:237] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:59.278908 20786 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b: soft_deleted_table [id=a9196b291b314bb0bcecf3f01e0c8e74] alter complete (version 1)
I20260709 13:18:59.290359 20786 catalog_manager.cc:5697] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 2 to 3, leader changed from 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65) to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 3 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.438791 21310 raft_consensus.cc:993] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578: : Instructing follower 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:59.439322 21261 raft_consensus.cc:1081] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Signalling peer 3106c79919c14b9b995f45270d557a1a to start an election
I20260709 13:18:59.440665 21040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3"
dest_uuid: "3106c79919c14b9b995f45270d557a1a"
 from {username='slave'} at 127.0.0.1:57530
I20260709 13:18:59.441089 21040 raft_consensus.cc:493] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:59.441352 21040 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.447700 21040 raft_consensus.cc:515] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:59.449672 21040 leader_election.cc:290] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 3 election: Requested vote from peers 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611), 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519)
I20260709 13:18:59.450311 20891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b97fb429b274b6087bc1de914d8f839"
I20260709 13:18:59.450968 20891 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.451665 20966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46ada670408442da2f3289f10ca33b3" candidate_uuid: "3106c79919c14b9b995f45270d557a1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "977832f370f745c7a0cb21e3dca7d578"
I20260709 13:18:59.452329 20966 raft_consensus.cc:3055] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:59.452607 20966 raft_consensus.cc:740] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 977832f370f745c7a0cb21e3dca7d578, State: Running, Role: LEADER
I20260709 13:18:59.453347 20966 consensus_queue.cc:260] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:59.454794 20966 raft_consensus.cc:3060] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:59.457418 20891 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 3.
I20260709 13:18:59.458572 21006 leader_election.cc:304] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3106c79919c14b9b995f45270d557a1a, 5b97fb429b274b6087bc1de914d8f839; no voters: 
I20260709 13:18:59.459249 21240 raft_consensus.cc:2804] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:59.459725 21240 raft_consensus.cc:697] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 3 LEADER]: Becoming Leader. State: Replica: 3106c79919c14b9b995f45270d557a1a, State: Running, Role: LEADER
I20260709 13:18:59.460719 20966 raft_consensus.cc:2468] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3106c79919c14b9b995f45270d557a1a in term 3.
I20260709 13:18:59.460553 21240 consensus_queue.cc:237] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } }
I20260709 13:18:59.468611 20786 catalog_manager.cc:5697] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a reported cstate change: term changed from 2 to 3, leader changed from 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66) to 3106c79919c14b9b995f45270d557a1a (127.10.4.67). New cstate: current_term: 3 leader_uuid: "3106c79919c14b9b995f45270d557a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3106c79919c14b9b995f45270d557a1a" member_type: VOTER last_known_addr { host: "127.10.4.67" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.705973 20891 raft_consensus.cc:1275] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:59.707151 21240 consensus_queue.cc:1048] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:59.710942 20966 raft_consensus.cc:1275] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:59.712765 21240 consensus_queue.cc:1048] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:59.867069 20891 raft_consensus.cc:1275] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:59.868356 21342 consensus_queue.cc:1048] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b97fb429b274b6087bc1de914d8f839" member_type: VOTER last_known_addr { host: "127.10.4.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:59.877023 20966 raft_consensus.cc:1275] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Refusing update from remote peer 3106c79919c14b9b995f45270d557a1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:59.879107 21284 consensus_queue.cc:1048] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "977832f370f745c7a0cb21e3dca7d578" member_type: VOTER last_known_addr { host: "127.10.4.66" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:00.245819 10257 tablet_server.cc:179] TabletServer@127.10.4.65:0 shutting down...
I20260709 13:19:00.288686 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:19:00.289388 10257 tablet_replica.cc:333] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.289987 10257 raft_consensus.cc:2243] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.290475 10257 raft_consensus.cc:2272] T 07e7f7268a2747c38dd024380d8dc95e P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.292424 10257 tablet_replica.cc:333] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.292981 10257 raft_consensus.cc:2243] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.293876 10257 raft_consensus.cc:2272] T a60a556fcad84768926201542f32dd63 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.295789 10257 tablet_replica.cc:333] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.296424 10257 raft_consensus.cc:2243] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.297281 10257 raft_consensus.cc:2272] T 62432463b07e4593be55c65c1ae972a7 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:00.297868 21006 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
I20260709 13:19:00.299191 10257 tablet_replica.cc:333] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.299872 10257 raft_consensus.cc:2243] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.300303 10257 raft_consensus.cc:2272] T e4e58e63b66b44b38d012f2e83ee8b3d P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:00.300727 21006 consensus_peers.cc:597] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a -> Peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Couldn't send request to peer 5b97fb429b274b6087bc1de914d8f839. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.302423 10257 tablet_replica.cc:333] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.303064 10257 raft_consensus.cc:2243] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.303602 10257 raft_consensus.cc:2272] T 5e1968ddaebf4232a8e0df881ee33b9f P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.306090 10257 tablet_replica.cc:333] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.306638 10257 raft_consensus.cc:2243] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.307353 10257 raft_consensus.cc:2272] T dfa44f2202b74d6fa6853bff5d64d1d2 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.309695 10257 tablet_replica.cc:333] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.310232 10257 raft_consensus.cc:2243] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.311148 10257 raft_consensus.cc:2272] T 986b76c2b9d545afa7c5f88eca7c5346 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.313216 10257 tablet_replica.cc:333] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.313794 10257 raft_consensus.cc:2243] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.314519 10257 raft_consensus.cc:2272] T c9682d66d5a14a50a243c111a280b42c P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.316437 10257 tablet_replica.cc:333] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.317003 10257 raft_consensus.cc:2243] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.317378 10257 raft_consensus.cc:2272] T 4761164a28c243c1ae1500d536445d9b P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.319454 10257 tablet_replica.cc:333] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.319993 10257 raft_consensus.cc:2243] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.320765 10257 raft_consensus.cc:2272] T d47c528029b94270a3e760ddcb2e6503 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.322856 10257 tablet_replica.cc:333] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.323452 10257 raft_consensus.cc:2243] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.323846 10257 raft_consensus.cc:2272] T aab942b20de44ae7bab51d2bf83142ac P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.326162 10257 tablet_replica.cc:333] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.326699 10257 raft_consensus.cc:2243] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.327116 10257 raft_consensus.cc:2272] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.328881 10257 tablet_replica.cc:333] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.329413 10257 raft_consensus.cc:2243] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.330180 10257 raft_consensus.cc:2272] T a8e2647fab8b4079861f111406365333 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.332175 10257 tablet_replica.cc:333] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.332669 10257 raft_consensus.cc:2243] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.333096 10257 raft_consensus.cc:2272] T b46ada670408442da2f3289f10ca33b3 P 5b97fb429b274b6087bc1de914d8f839 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.335980 10257 tablet_replica.cc:333] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.336546 10257 raft_consensus.cc:2243] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.336990 10257 raft_consensus.cc:2272] T a5d7f0637d564294bcd6920033ba9fbf P 5b97fb429b274b6087bc1de914d8f839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.339226 10257 tablet_replica.cc:333] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
W20260709 13:19:00.339280 20932 consensus_peers.cc:597] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 -> Peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Couldn't send request to peer 5b97fb429b274b6087bc1de914d8f839. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.339764 10257 raft_consensus.cc:2243] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.340281 10257 raft_consensus.cc:2272] T e83e6552f0674e9698832dfbd218b118 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.342639 10257 tablet_replica.cc:333] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.343217 10257 raft_consensus.cc:2243] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.343729 10257 raft_consensus.cc:2272] T 7835def769884fc69be515fa1c172ae8 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.345686 10257 tablet_replica.cc:333] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839: stopping tablet replica
I20260709 13:19:00.346163 10257 raft_consensus.cc:2243] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.346572 10257 raft_consensus.cc:2272] T 2f1e56bf4cc5416ea22348bc387b57a9 P 5b97fb429b274b6087bc1de914d8f839 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:00.365792 20932 consensus_peers.cc:597] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 -> Peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Couldn't send request to peer 5b97fb429b274b6087bc1de914d8f839. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.385855 10257 tablet_server.cc:196] TabletServer@127.10.4.65:0 shutdown complete.
I20260709 13:19:00.419373 10257 tablet_server.cc:179] TabletServer@127.10.4.66:0 shutting down...
W20260709 13:19:00.422060 20932 consensus_peers.cc:597] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 -> Peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Couldn't send request to peer 5b97fb429b274b6087bc1de914d8f839. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:41611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.443826 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:19:00.444728 10257 tablet_replica.cc:333] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.445329 10257 raft_consensus.cc:2243] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.445956 10257 raft_consensus.cc:2272] T 07e7f7268a2747c38dd024380d8dc95e P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:00.447623 20932 consensus_peers.cc:597] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 -> Peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Couldn't send request to peer 5b97fb429b274b6087bc1de914d8f839. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:41611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.448351 10257 tablet_replica.cc:333] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.448974 10257 raft_consensus.cc:2243] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.449390 10257 raft_consensus.cc:2272] T a60a556fcad84768926201542f32dd63 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.451205 10257 tablet_replica.cc:333] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.451812 10257 raft_consensus.cc:2243] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.452208 10257 raft_consensus.cc:2272] T 62432463b07e4593be55c65c1ae972a7 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.453966 10257 tablet_replica.cc:333] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.454488 10257 raft_consensus.cc:2243] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.455181 10257 raft_consensus.cc:2272] T 5e1968ddaebf4232a8e0df881ee33b9f P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.456946 10257 tablet_replica.cc:333] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.457463 10257 raft_consensus.cc:2243] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.458163 10257 raft_consensus.cc:2272] T e4e58e63b66b44b38d012f2e83ee8b3d P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.459986 10257 tablet_replica.cc:333] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.460521 10257 raft_consensus.cc:2243] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.460979 10257 raft_consensus.cc:2272] T dfa44f2202b74d6fa6853bff5d64d1d2 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.463271 10257 tablet_replica.cc:333] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.463996 10257 raft_consensus.cc:2243] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.464509 10257 raft_consensus.cc:2272] T 986b76c2b9d545afa7c5f88eca7c5346 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.466679 10257 tablet_replica.cc:333] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
W20260709 13:19:00.466948 20932 consensus_peers.cc:597] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 -> Peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Couldn't send request to peer 5b97fb429b274b6087bc1de914d8f839. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:41611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.467391 10257 raft_consensus.cc:2243] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.467936 10257 raft_consensus.cc:2272] T c9682d66d5a14a50a243c111a280b42c P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.469738 10257 tablet_replica.cc:333] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.470254 10257 raft_consensus.cc:2243] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.470975 10257 raft_consensus.cc:2272] T 4761164a28c243c1ae1500d536445d9b P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.472919 10257 tablet_replica.cc:333] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.473389 10257 raft_consensus.cc:2243] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.474049 10257 raft_consensus.cc:2272] T 7835def769884fc69be515fa1c172ae8 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.475759 10257 tablet_replica.cc:333] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.476266 10257 raft_consensus.cc:2243] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.477069 10257 raft_consensus.cc:2272] T 2f1e56bf4cc5416ea22348bc387b57a9 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.479050 10257 tablet_replica.cc:333] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.479626 10257 raft_consensus.cc:2243] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.480032 10257 raft_consensus.cc:2272] T aab942b20de44ae7bab51d2bf83142ac P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.482400 10257 tablet_replica.cc:333] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.482970 10257 raft_consensus.cc:2243] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.483366 10257 raft_consensus.cc:2272] T d47c528029b94270a3e760ddcb2e6503 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.485359 10257 tablet_replica.cc:333] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.485858 10257 raft_consensus.cc:2243] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.486624 10257 raft_consensus.cc:2272] T a5d7f0637d564294bcd6920033ba9fbf P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.488579 10257 tablet_replica.cc:333] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.489054 10257 raft_consensus.cc:2243] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.489431 10257 raft_consensus.cc:2272] T b46ada670408442da2f3289f10ca33b3 P 977832f370f745c7a0cb21e3dca7d578 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.491783 10257 tablet_replica.cc:333] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.492266 10257 raft_consensus.cc:2243] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.492690 10257 raft_consensus.cc:2272] T e83e6552f0674e9698832dfbd218b118 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.494560 10257 tablet_replica.cc:333] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.494989 10257 raft_consensus.cc:2243] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.495728 10257 raft_consensus.cc:2272] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 977832f370f745c7a0cb21e3dca7d578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.497346 10257 tablet_replica.cc:333] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578: stopping tablet replica
I20260709 13:19:00.497874 10257 raft_consensus.cc:2243] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.498288 10257 raft_consensus.cc:2272] T a8e2647fab8b4079861f111406365333 P 977832f370f745c7a0cb21e3dca7d578 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:00.502449 21006 consensus_peers.cc:597] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a -> Peer 5b97fb429b274b6087bc1de914d8f839 (127.10.4.65:41611): Couldn't send request to peer 5b97fb429b274b6087bc1de914d8f839. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.65:41611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.536386 10257 tablet_server.cc:196] TabletServer@127.10.4.66:0 shutdown complete.
I20260709 13:19:00.564739 10257 tablet_server.cc:179] TabletServer@127.10.4.67:0 shutting down...
I20260709 13:19:00.590992 10257 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:19:00.591684 10257 tablet_replica.cc:333] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.592259 10257 raft_consensus.cc:2243] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.593042 10257 raft_consensus.cc:2272] T 07e7f7268a2747c38dd024380d8dc95e P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.595009 10257 tablet_replica.cc:333] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.595609 10257 raft_consensus.cc:2243] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.596014 10257 raft_consensus.cc:2272] T 62432463b07e4593be55c65c1ae972a7 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.597733 10257 tablet_replica.cc:333] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.598217 10257 raft_consensus.cc:2243] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.598611 10257 raft_consensus.cc:2272] T c9682d66d5a14a50a243c111a280b42c P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.600345 10257 tablet_replica.cc:333] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.600841 10257 raft_consensus.cc:2243] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.601234 10257 raft_consensus.cc:2272] T a60a556fcad84768926201542f32dd63 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.602883 10257 tablet_replica.cc:333] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.603387 10257 raft_consensus.cc:2243] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.603809 10257 raft_consensus.cc:2272] T 5e1968ddaebf4232a8e0df881ee33b9f P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.605468 10257 tablet_replica.cc:333] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.605960 10257 raft_consensus.cc:2243] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.606360 10257 raft_consensus.cc:2272] T e4e58e63b66b44b38d012f2e83ee8b3d P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.608258 10257 tablet_replica.cc:333] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.608850 10257 raft_consensus.cc:2243] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.609251 10257 raft_consensus.cc:2272] T dfa44f2202b74d6fa6853bff5d64d1d2 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:00.610406 21004 consensus_peers.cc:597] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a -> Peer 977832f370f745c7a0cb21e3dca7d578 (127.10.4.66:43519): Couldn't send request to peer 977832f370f745c7a0cb21e3dca7d578. Status: Network error: Client connection negotiation failed: client connection to 127.10.4.66:43519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.611356 10257 tablet_replica.cc:333] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.612012 10257 raft_consensus.cc:2243] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.612421 10257 raft_consensus.cc:2272] T 986b76c2b9d545afa7c5f88eca7c5346 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.614238 10257 tablet_replica.cc:333] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.614801 10257 raft_consensus.cc:2243] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.615549 10257 raft_consensus.cc:2272] T aab942b20de44ae7bab51d2bf83142ac P 3106c79919c14b9b995f45270d557a1a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.617805 10257 tablet_replica.cc:333] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.618322 10257 raft_consensus.cc:2243] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.618733 10257 raft_consensus.cc:2272] T d47c528029b94270a3e760ddcb2e6503 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.620744 10257 tablet_replica.cc:333] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.621277 10257 raft_consensus.cc:2243] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.621711 10257 raft_consensus.cc:2272] T 7835def769884fc69be515fa1c172ae8 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.623368 10257 tablet_replica.cc:333] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.623904 10257 raft_consensus.cc:2243] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.624307 10257 raft_consensus.cc:2272] T 2f1e56bf4cc5416ea22348bc387b57a9 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.626194 10257 tablet_replica.cc:333] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.626682 10257 raft_consensus.cc:2243] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.627066 10257 raft_consensus.cc:2272] T a5d7f0637d564294bcd6920033ba9fbf P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.629043 10257 tablet_replica.cc:333] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.629508 10257 raft_consensus.cc:2243] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.630215 10257 raft_consensus.cc:2272] T b46ada670408442da2f3289f10ca33b3 P 3106c79919c14b9b995f45270d557a1a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.632819 10257 tablet_replica.cc:333] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.633311 10257 raft_consensus.cc:2243] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.633698 10257 raft_consensus.cc:2272] T 4761164a28c243c1ae1500d536445d9b P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.635591 10257 tablet_replica.cc:333] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.636042 10257 raft_consensus.cc:2243] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.636806 10257 raft_consensus.cc:2272] T e83e6552f0674e9698832dfbd218b118 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.638757 10257 tablet_replica.cc:333] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.639230 10257 raft_consensus.cc:2243] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.639644 10257 raft_consensus.cc:2272] T 43fa8704ba3c47c6a962f4b9a36b8c74 P 3106c79919c14b9b995f45270d557a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.641327 10257 tablet_replica.cc:333] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a: stopping tablet replica
I20260709 13:19:00.641800 10257 raft_consensus.cc:2243] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:00.642179 10257 raft_consensus.cc:2272] T a8e2647fab8b4079861f111406365333 P 3106c79919c14b9b995f45270d557a1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.680585 10257 tablet_server.cc:196] TabletServer@127.10.4.67:0 shutdown complete.
I20260709 13:19:00.708884 10257 master.cc:562] Master@127.10.4.126:45619 shutting down...
I20260709 13:19:00.727954 10257 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:19:00.728472 10257 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:00.728783 10257 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4442319c201049d98bc8b46cb5407e9b: stopping tablet replica
I20260709 13:19:00.747922 10257 master.cc:584] Master@127.10.4.126:45619 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11110 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (23389 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (305684 ms total)
[  PASSED  ] 8 tests.
I20260709 13:19:00.795707 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 395 messages since previous log ~14 seconds ago
I20260709 13:19:00.795907 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 395 messages since previous log ~14 seconds ago
I20260709 13:19:00.796041 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 393 messages since previous log ~14 seconds ago
I20260709 13:19:00.796218 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~31 seconds ago
I20260709 13:19:00.796362 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~31 seconds ago
I20260709 13:19:00.796519 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 5 messages since previous log ~136 seconds ago
I20260709 13:19:00.796705 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~0 seconds ago
I20260709 13:19:00.796937 10257 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~5 seconds ago
